How to fix GPG error for extras.ubuntu.com oneiric Release?

Add new key

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 16126D3A3E5C1192
sudo apt-key del 16126D3A3E5C1192 #delete Key

list all public key with PPA

sudo apt-key finger
apt-get clean # Remove cached packages 
cd /var/lib/apt 
mv lists lists.old # Backup mirror info 
mkdir -p lists/partial # Recreate directory structure 
apt-get clean 
apt-get update # Fetch mirror info

How to check mail server

1. get the MX record:

dig MX domainname.com

or http://www.mxtoolbox.com

2. connect to the server with telnet

> telnet ASPMX.L.GOOGLE.COM 25
Trying 74.125.39.27...
Connected to ASPMX.L.GOOGLE.COM.
Escape character is '^]'.
220 mx.google.com ESMTP y26si6167249fag.156
helo mydomain.com
250 mx.google.com at your service
mail from: <me@mydomain.com>
250 2.1.0 OK y26si6167249fag.156
rcpt to: <bogus@pentest.ro>
550-5.1.1 The email account that you tried to reach does not exist. Please try
550-5.1.1 double-checking the recipient's email address for typos or
550-5.1.1 unnecessary spaces. Learn more at
550 5.1.1 http://mail.google.com/support/bin/answer.py?answer=6596 y26si6167249fag.156
rcpt to: <somevalidaddress@pentest.ro>
250 2.1.5 OK y26si6167249fag.156
quit
221 2.0.0 closing connection y26si6167249fag.156
Connection closed by foreign host.

The bolded lines are the ones you type, the others are responses from the server.

The most common command in Vim, every programmer should know them :)

I have to clarify that all these following commands that I list here are copied from web.

http://www.cs.swarthmore.edu/help/vim/home.html

Basics

:e filename Open filename for edition
:w Save file
:q Exit Vim
:w! Exit Vim without saving

Search

/word Search word from top to bottom
?word Search word from bottom to top
/jo[ha]n Search john or joan
/\< the Search the, theatre or then
/the\> Search the or breathe
/\< the\> Search the
/\< ¦.\> Search all words of 4 letters
/\/ Search fred but not alfred or frederick
/fred\|joe Search fred or joe
/\<\d\d\d\d\> Search exactly 4 digits
/^\n\{3} Find 3 empty lines
:bufdo /searchstr/ Search in all open files

Replace

:%s/old/new/g Replace all occurences of old by new in file
:%s/old/new/gw Replace all occurences with confirmation
:2,35s/old/new/g Replace all occurences between lines 2 and 35
:5,$s/old/new/g Replace all occurences from line 5 to EOF
:%s/^/hello/g Replace the begining of each line by hello
:%s/$/Harry/g Replace the end of each line by Harry
:%s/onward/forward/gi Replace onward by forward, case unsensitive
:%s/ *$//g Delete all white spaces
:g/string/d Delete all lines containing string
:v/string/d Delete all lines containing which didn’t contain string
:s/Bill/Steve/ Replace the first occurence of Bill by Steve in current line
:s/Bill/Steve/g Replace Bill by Steve in current line
:%s/Bill/Steve/g Replace Bill by Steve in all the file
:%s/\r//g Delete DOS carriage returns (^M)
:%s/\r/\r/g Transform DOS carriage returns in returns
:%s#<[^>]\+>##g Delete HTML tags but keeps text
:%s/^\(.*\)\n\1$/\1/ Delete lines which appears twice
Ctrl+a Increment number under the cursor
Ctrl+x Decrement number under cursor
ggVGg? Change text to Rot13

Case

Vu Lowercase line
VU Uppercase line
g~~ Invert case
vEU Switch word to uppercase
vE~ Modify word case
ggguG Set all text to lowercase
:set ignorecase Ignore case in searches
:set smartcase Ignore case in searches excepted if an uppercase letter is used
:%s/\<./\u&/g Sets first letter of each word to uppercase
:%s/\<./\l&/g Sets first letter of each word to lowercase
:%s/.*/\u& Sets first letter of each line to uppercase
:%s/.*/\l& Sets first letter of each line to lowercase

Read/Write files

:1,10 w outfile Saves lines 1 to 10 in outfile
:1,10 w >> outfile Appends lines 1 to 10 to outfile
:r infile Insert the content of infile
:23r infile Insert the content of infile under line 23

File explorer

:e . Open integrated file explorer
:Sex Split window and open integrated file explorer
:browse e Graphical file explorer
:ls List buffers
:cd .. Move to parent directory
:args List files
:args *.php Open file list
:grep expression *.php Returns a list of .php files contening expression
gf Open file name under cursor

Interact with Unix

:!pwd Execute the pwd unix command, then returns to Vi
!!pwd Execute the pwd unix command and insert output in file
:sh Temporary returns to Unix
$exit Retourns to Vi

Alignment

:%!fmt Align all lines
!}fmt Align all lines at the current position
5!!fmt Align the next 5 lines

Tabs

:tabnew Creates a new tab
gt Show next tab
:tabfirst Show first tab
:tablast Show last tab
:tabm n(position) Rearrange tabs
:tabdo %s/foo/bar/g Execute a command in all tabs
:tab ball Puts all open files in tabs

Window spliting

:e filename Edit filename in current window
:split filename Split the window and open filename
ctrl-w up arrow Puts cursor in top window
ctrl-w ctrl-w Puts cursor in next window
ctrl-w_ Maximise current window
ctrl-w= Gives the same size to all windows
10 ctrl-w+ Add 10 lines to current window
:vsplit file Split window vertically
:sview file Same as :split in readonly mode
:hide Close current window
:­nly Close all windows, excepted current
:b 2 Open #2 in this window

Auto-completion

Ctrl+n Ctrl+p (in insert mode) Complete word
Ctrl+x Ctrl+l Complete line
:set dictionary=dict Define dict as a dictionnary
Ctrl+x Ctrl+k Complete with dictionnary

Marks

mk Marks current position as k
˜k Moves cursor to mark k
d™k Delete all until mark k

Abbreviations

:ab mail mail@provider.org Define mail as abbreviation of mail@provider.org

Text indent

:set autoindent Turn on auto-indent
:set smartindent Turn on intelligent auto-indent
:set shiftwidth=4 Defines 4 spaces as indent size
ctrl-t, ctrl-d Indent/un-indent in insert mode
>> Indent
<< Un-indent

Syntax highlighting

:syntax on Turn on syntax highlighting
:syntax off Turn off syntax highlighting
:set syntax=perl Force syntax highlighting

JAVA I/O

按照《Unix网络编程》的划分,IO模型可以分为:阻塞IO、非阻塞IO、IO复用、信号驱动IO和异步IO

按照POSIX标准来划分只分为两类:同步IO和异步IO.

如何区分同步IO和异步IO:

首先一个IO操作其实分成了两个步骤:发起IO请求和实际的IO操作

  • 同步IO和异步IO的区别就在于第二个步骤是否阻塞,如果实际的IO读写阻塞请求进程,那么就是同步IO,因此阻塞IO、非阻塞IO、IO复用、信号驱动IO都是同步IO
  • 如果不阻塞,而是操作系统帮你做完IO操作再将结果返回给你,那么就是异步IO。
  • 阻塞IO和非阻塞IO的区别在于第一步,发起IO请求是否会被阻塞,如果阻塞直到完成那么就是传统的阻塞IO,如果不阻塞,那么就是非阻塞IO。

JAVA的传统流:

流是一个抽象的概念。当Java程序需要从数据源读取数据时,会开启一个到数据源的流。数据源可以是文件,内存或者网络等。同样,当程序需要输出数据到目的地时也一样会开启一个流,数据目的地也可以是文件、内存或者网络等。流的创建是为了更方便地处理数据的输入输出。

字节流: InputStream && OutPutStream

字节流的处理类有很多,他们都继承自InputStream或者OutputStream抽象类。

//InputStream:
read();
read(byte[] b) ;
read(byte[] b, int off, int len) ;

//OutputStream:
write(int b);
write(byte[] b);
write(byte[] b, int off, int len)

字符流:Reader && Writer

所有的字符流操作类都继承自Reader或者Writer这两个抽象类。

NIO:

The most important distinction between the original I/O library (found in java.io.*)
and NIO has to do with how data is packaged and transmitted. As previously
mentioned, original I/O deals with data in streams, whereas NIO deals with data in
blocks.

JAVA socket

Asynchronous I/O is a method for reading and writing data without blocking.

Asynchronous I/O calls, on the other hand, do not block. Instead, you register your
interest in a particular I/O event — the arrival of readable data, a new socket
connection, and so on — and the system tells you when such an event occurs.

One of the advantages of asynchronous I/O is that it lets you do I/O from a great many inputs and outputs at the same time. Synchronous programs often have to resort to polling, or to the creation of many, many threads, to deal with lots of connections. With asynchronous I/O, you can listen for I/O events on an arbitrary number of channels, without polling and without extra threads.

Selector:

The central object in asynchronous I/O is called the Selector. A Selector is where
you register your intere.

ServerSocketChannel:

In order to receive connections, we need a ServerSocketChannel. In fact, we need
one for each of the ports on which we are going to listen

JAVA Thread

线程的状态有:new、runnable、running、waiting、timed_waiting、blocked、dead

http://www.iteye.com/topic/1119957

  • wait( ) tells the calling thread to give up the monitor and go to sleep until some other
    thread enters the same monitor and calls notify( ).
  • notify( ) wakes up the first thread that called wait( ) on the same object.
  • notifyAll( ) wakes up all the threads that called wait( ) on the same object. The
    highest priority thread will run first.

Interfaces Hash table Implementations Resizable array Implementations Tree Implementations Linked list Implementations Hash table + Linked list Implementations
Set HashSet TreeSet LinkedHashSet
List ArrayList LinkedList
Queue
Map HashMap TreeMap LinkedHashMap

Introduce to Vim/Vi

The only reason that motivated me to learn Vim is sometimes I have to programming or debugging on production server. I understand using advanced IDE such as eclipse, NetBeans etc makes our life much more easier, however, knowing how to use Vi/Vim still will be a handy tools for us.

  • Installation

    sudo apt-get install vim

    or go to official site to download.

  • setting Environment

    We could create an config file for Vim.
    There is vim config as sample.

    if we want to apply environment to all user account, we should put the option file in global location.
    /usr/local/vim/vimrc

    If we want settings specific to a certain user then create a config file in their home directory.
    /home/username/.vimrc

  • Switch between different modes

    There are three modes in Vim.
    Command mode: all keystrokes are interpreted as commands.
    Insert mode: most keystrokes are inserted as text (leaving out those with modifier keys).
    Visual mode: helps to visually select some text, may be seen as a submode of the the command mode.

     

Set Hadoop development environment

Preparation

  • set up Java environment
  • download hadoop
  • set up maven
  • set up Pseudo-Distributed mode

PS: this is the Pseudo-Distributed mode setting up: hadoop tutorial

hadoop project with MVN

  • Create a java project hierarchy project:

    mvn archetype:create -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId={your groupId here} -DartifactId={your artifactId here}
    

  • Generate eclipe project file:

    mvn eclipse:eclipse
    
  • Add hadoop dependency to POX file

    <dependency>
         <groupId>org.apache.hadoop</groupId>
         <artifactId>hadoop-core</artifactId>
         <version>0.20.2</version>
    </dependency>
    

  • put all jars into target/dependencies

    mvn dependency:copy-dependencies
    
  • package mapReducer into jar

    mvn jar:jar
    

Hadoop command

# will print all the command that you can execute on the HDFS
hadoop fs -help
# will perform an ls from the HDFS home directory.
hadoop fs -ls
# will create the directory 'input' in the HDFS home directory (check #if it does not already exists)
hadoop fs -mkdir input
# will create the directory 'input' in the HDFS home directory (check #if it does not already exists)
hadoop fs -mkdir output
# will put your local copy of mobyDick into the directory 'input' on # the HDFS
hadoop fs -put mobyDick.txt input
hadoop jar {yourjarhere}.jar {packagenem.ClassName} input output

Understanding Google app datastorage 1st

http://ikaisays.com/2010/06/10/using-the-bulkloader-with-java-app-engine/

 necessary concept
Entity && Properties

A data object in the App Engine datastore is known as an entity. Entity have one or more properties. (like table and column in BD)

However, an entities may have different properties. And a propert can have more than one value with mixed type.

A query on a property with multiple values tests whether any of the values meets the query criteria. This makes such properties useful for testing for membership.

Transactions and Entity Groups (basic CRUD)

Entity groups are defined by a hierarchy of relationships between entities. (parent-children relationship tree).

Each entity has a path of parent-child relationships from a root entity to itself (the shortest path being no parent). This path is an essential part of the entity’s complete key. A complete key can be represented by the kind and ID or key name of each entity in the path.

optimistic concurrency: While one app instance is applying changes to entities in an entity group, all other attempts to update the group, either by updating existing entities or creating new entities, fail on commit.  using entity groups limits the number of concurrent writes you can do on any entity in that group.

Queries and Indexes

App Engine application defines its indexes in a configuration file (although indexes for some types of queries are provided automatically).

Differences From SQL

 

Because all queries on App Engine are served by pre-built indexes, the types of queries that can be executed are more restrictive than those allowed on a relational database with SQL. No joins are supported in the datastore. The datastore also does not allow inequality filtering on multiple properties or filtering of data based on results of a sub-query.