RabbitMQ : Getting Started.

RabbitMQ libraries

RabbitMQ speaks a protocol called AMQP. To use Rabbit you’ll need a library that understands the same protocol as Rabbit. There is a choice of libraries for almost every programming language. For python it’s no different and there are a bunch of libraries to choose from:

Here we use pika. To install it you can use the pip package management tool:

$ sudo pip install pika==0.9.8

The installation depends on pip and git-core packages, you may need to install them first.

  • On Ubuntu:
    $ sudo apt-get install python-pip git-core
  • On Debian:
    $ sudo apt-get install python-setuptools git-core
    $ sudo easy_install pip
  • On Windows: To install easy_install, run the MS Windows Installer for setuptools
    > easy_install pip
    > pip install pika==0.9.8

Now, to as first step towards starting to use RabbitMQ you can have a look how this works.

These programs are analogous to any hello-world program we write in any other language.

Send_Msg.py : This python program works like a Producer. It sends a message on to RabbitMQ.It connects to the default Exchange and creates a queue called and then sends a message to the queue.

Recieve_Msg,py : This program shall consume the messages from the Hello_World named Queue.

At any time to see the queues created and the number of messages in the queue currently, run

 rabbitmqctl list_queues

How to Run : You can run one “python Send_Msg.py <your_message>” repetitively from one terminal and runRecieve_Msg.py in two or more different terminals and it would give you a clear understanding of how messages are being sent and recieved. Source : RabbitMQ Documentation.

Advertisements

Git – Bare Minimum you need to know.

Git – A Quick Start

The other Post explains some of the basic concepts of Git and SCM. This post is primarily aimed at explaining to you some of the git commands that you would use most often.

Git Lingo

Repositories

Git Repositories/repos are basically referred to a complete bundle of Code+metadata (history of commits and logs). Advantages of git over other SCMs like SVN is that it is a Distributed Version Control System . This in simple terms mean that the metadata would reside with the code and would get copied where-ever you clone the code and hence in a synced condition, the system that has cloned the repository would have the same amount of meta-data and info which the the server from which the cloned from had, When you clone a repository, the complete history/metadata gets copied along with the code, so every working copy has the complete information from Commit 1 to the latest commit into the repo.

Branches

Branches allow parallel development of code for two different threads in the same repository. There could be several Use-Cases requiring this like, if you have a current stable-release branch on which you are fixing bugs and and it out for consumers and could have a separate branch for future development .  Commits onto any particular branch would affect the code onto only that branch and keep all the rest of the banches untouched. By-default, there is a master branch, which gets created initially. You may fork out a branch out of any given branch give it a name, continue your development, and finally merge the branch back to the branch from which you had initially forked the branch.

One good tool to view all the branching in a repo graphically is by using a utility called gitk. gitk would graphically represent the branch layout and point your location in the tree.

Tags

Tags can be compared to some specific milestone in  the commit-history of the code. Stable-releases, release-versions can be be used as tags. Basically, a tag would associate one specific commit-id with some logical-tag-name. Developers can identity any  milestone-release, by listing the tags in the git repositories and can get exact version of code by checking out that particular tag/commit=id.

Commit

Committing a code is analogous to saving any file to your local system after making some changes to it. Each time you execute a git commit command. All the delta-changes from the previous commit are calculated and added to the git history and commit-id is generated in return. Also, it is a good practice to put a message in each of the commits explaining the logic behind you  making all the given changes to the file/files

GitHub

GitHub is a web-based hosting service for software development projects that use the Git revision control system. GitHub offers both paid plans for private repositories, and free accounts for open source projects. As of May 2011, GitHub was the most popular open source code repository site. To avoid any confusions, github is a site offering service based on Git.

Basic Commands

===================

Git init

What is does ?

Creates an empty Git repository or reinitialize an existing one.

How  it does ?

This command creates an empty Git repository – basically a .git directory with subdirectories forobjectsrefs/headsrefs/tags, and template files. An initial HEAD file that references the HEAD of the master branch is also created.

       If the $GIT_DIR environment variable is set then it specifies a path to use instead of ./.git for the base of the repository.

       If the object storage directory is specified via the $GIT_OBJECT_DIRECTORY environment variable then the sha1 directories are created underneath – otherwise the default $GIT_DIR/objects directory is used.

        Running git init in an existing repository is safe. It will not overwrite things that are already there. The primary reason for rerunning git init is to pick up newly added templates (or to move the repository to another place if –separate-git-dir is given).

How do I use it?

Start a new Git repository for an existing code base

$ cd /path/to/my/codebase $ git init <1> $ git add . <2>
So two steps would be as follows :
  1. prepare /path/to/my/codebase/.git directory
  2. add all existing file to the index

To learn about all the options in details, you can run

 $ man git init

Git Clone

What is does ?

Clone a repository into a new directory.

How  it does ?

Clones a repository into a newly created directory, creates remote-tracking branches for each branch in the cloned repository (visible using git branch -r), and creates and checks out an initial branch that is forked from the cloned repository’s currently active branch.

After the clone, a plain git fetch without arguments will update all the remote-tracking branches, and a git pull without arguments will in addition merge the remote master branch into the current master branch, if any (this is untrue when “–single-branch” is given; see below).

This default configuration is achieved by creating references to the remote branch heads underrefs/remotes/origin and by initializing remote.origin.url and remote.origin.fetchconfiguration variables.

How do I use it?

Clone from upstream:

$ git clone git://git.kernel.org/pub/scm/.../linux-2.6 my2.6
$ cd my2.6
$ make
Make a local clone that borrows from the current directory, without checking things out:
$ git clone -l -s -n . ../copy
$ cd ../copy
$ git show-branch

Clone from upstream while borrowing from an existing local directory:

$ git clone --reference my2.6 \
	git://git.kernel.org/pub/scm/.../linux-2.7 \
	my2.7
$ cd my2.7

Create a bare repository to publish your changes to the public:
$ git clone --bare -l /home/proj/.git /pub/scm/proj.git

Create a repository on the kernel.org machine that borrows from Linus:
$ git clone --bare -l -s /pub/scm/.../torvalds/linux-2.6.git \
    /pub/scm/.../me/subsys-2.6.git

To learn about all the options in details, you can run

 $ man git clone

Git Pull

What is does ?

Fetches from and merges with another repository or a local branch

How  it does ?

Incorporates changes from a remote repository into the current branch. In its default mode, git pull is shorthand for git fetch followed by git merge FETCH_HEAD.

More precisely, git pull runs git fetch with the given parameters and calls git merge to merge the retrieved branch heads into the current branch. With --rebase, it runs git rebase instead of git merge.

<repository> should be the name of a remote repository as passed to git-fetch(1). <refspec> can name an arbitrary remote ref (for example, the name of a tag) or even a collection of refs with corresponding remote-tracking branches (e.g., refs/heads/*:refs/remotes/origin/*), but usually it is the name of a branch in the remote repository.

Default values for <repository> and <branch> are read from the “remote” and “merge” configuration for the current branch as set by git-branch(1) --track.

Assume the following history exists and the current branch is “master“:

	  A---B---C master on origin
	 /
    D---E---F---G master

Then “git pull” will fetch and replay the changes from the remote master branch since it diverged from the local master (i.e., E) until its current commit (C) on top of master and record the result in a new commit along with the names of the two parent commits and a log message from the user describing the changes.

	  A---B---C remotes/origin/master
	 /         \
    D---E---F---G---H master

See git-merge(1) for details, including how conflicts are presented and handled.

In Git 1.7.0 or later, to cancel a conflicting merge, use git reset --mergeWarning: In older versions of Git, running git pull with uncommitted changes is discouraged: while possible, it leaves you in a state that may be hard to back out of in the case of a conflict.

If any of the remote changes overlap with local uncommitted changes, the merge will be automatically cancelled and the work tree untouched. It is generally best to get any local changes in working order before pulling or stash them away with git-stash(1).
How do I use it?

  • Update the remote-tracking branches for the repository you cloned from, then merge one of them into your current branch:
    $ git pull, git pull origin 

    Normally the branch merged in is the HEAD of the remote repository, but the choice is determined by the branch.<name>.remote and branch.<name>.merge options; see git-config(1) for details. 

  • Merge into the current branch the remote branch next:
    $ git pull origin next 

    This leaves a copy of next temporarily in FETCH_HEAD, but does not update any remote-tracking branches. Using remote-tracking branches, the same can be done by invoking fetch and merge:

    $ git fetch origin
    $ git merge origin/next 

If you tried a pull which resulted in complex conflicts and would want to start over, you can recover with git reset.

To learn about all the options in details, you can run

 $ man git pull 

Git Checkout

What is does ?

Checkout a branch or paths to the working tree

How  it does ?

Updates files in the working tree to match the version in the index or the specified tree. If no paths are given, git checkout will also update HEAD to set the specified branch as the current branch.

git checkout <branch>
To prepare for working on <branch>, switch to it by updating the index and the files in the working tree, and by pointing HEAD at the branch. Local modifications to the files in the working tree are kept, so that they can be committed to the <branch>.

If <branch> is not found but there does exist a tracking branch in exactly one remote (call it <remote>) with a matching name, treat as equivalent to

$ git checkout -b <branch> --track <remote>/<branch>

You could omit <branch>, in which case the command degenerates to “check out the current branch”, which is a glorified no-op with a rather expensive side-effects to show only the tracking information, if exists, for the current branch.

git checkout -b|-B <new_branch> [<start point>]
Specifying -b causes a new branch to be created as if git-branch(1) were called and then checked out. In this case you can use the --track or --no-track options, which will be passed to git branch. As a convenience, --track without -b implies branch creation; see the description of --track below.

If -B is given, <new_branch> is created if it doesn’t exist; otherwise, it is reset. This is the transactional equivalent of

$ git branch -f <branch> [<start point>]
$ git checkout <branch>

that is to say, the branch is not reset/created unless “git checkout” is successful.

git checkout –detach [<branch>]
git checkout <commit>
Prepare to work on top of <commit>, by detaching HEAD at it (see “DETACHED HEAD” section), and updating the index and the files in the working tree. Local modifications to the files in the working tree are kept, so that the resulting working tree will be the state recorded in the commit plus the local modifications.

Passing --detach forces this behavior in the case of a <branch> (without the option, giving a branch name to the command would check out the branch, instead of detaching HEAD at it), or the current commit, if no <branch> is specified.

git checkout [-p|–patch] [<tree-ish>] [–] <pathspec>…
When <paths> or --patch are given, git checkout does not switch branches. It updates the named paths in the working tree from the index file or from a named <tree-ish> (most often a commit). In this case, the -b and --track options are meaningless and giving either of them results in an error. The <tree-ish> argument can be used to specify a specific tree-ish (i.e. commit, tag or tree) to update the index for the given paths before updating the working tree.

The index may contain unmerged entries because of a previous failed merge. By default, if you try to check out such an entry from the index, the checkout operation will fail and nothing will be checked out. Using -f will ignore these unmerged entries. The contents from a specific side of the merge can be checked out of the index by using --ours or --theirs. With -m, changes made to the working tree file can be discarded to re-create the original conflicted merge result.

How do I use it?
The following sequence checks out the master branch, reverts the Makefile to two revisions back, deletes hello.c by mistake, and gets it back from the index.

$ git checkout master
$ git checkout master~2 Makefile
$ rm -f hello.c
$ git checkout hello.c
switch branch

take a file out of another commit

restore hello.c from the index

If you want to check out all C source files out of the index, you can say

$ git checkout — ‘*.c’
Note the quotes around *.c. The file hello.c will also be checked out, even though it is no longer in the working tree, because the file globbing is used to match entries in the index (not in the working tree by the shell).

If you have an unfortunate branch that is named hello.c, this step would be confused as an instruction to switch to that branch. You should instead write:

$ git checkout — hello.c
After working in the wrong branch, switching to the correct branch would be done using:

$ git checkout mytopic
However, your “wrong” branch and correct “mytopic” branch may differ in files that you have modified locally, in which case the above checkout would fail like this:

$ git checkout mytopic
error: You have local changes to ‘frotz’; not switching branches.
You can give the -m flag to the command, which would try a three-way merge:

$ git checkout -m mytopic
Auto-merging frotz
After this three-way merge, the local modifications are not registered in your index file, so git diff would show you what changes you made since the tip of the new branch.

When a merge conflict happens during switching branches with the -m option, you would see something like this:

$ git checkout -m mytopic
Auto-merging frotz
ERROR: Merge conflict in frotz
fatal: merge program failed
At this point, git diff shows the changes cleanly merged as in the previous example, as well as the changes in the conflicted files. Edit and resolve the conflict and mark it resolved with git add as usual:

$ edit frotz
$ git add frotz

To learn about all the options in details, you can run

$ man git checkout

Git Add

What is does ?

Adds file contents to the index

How  it does ?

This command updates the index using the current content found in the working tree, to prepare the content staged for the next commit. It typically adds the current content of existing paths as a whole, but with some options it can also be used to add content with only part of the changes made to the working tree files applied, or remove paths that do not exist in the working tree anymore.

The “index” holds a snapshot of the content of the working tree, and it is this snapshot that is taken as the contents of the next commit. Thus after making any changes to the working directory, and before running the commit command, you must use the add command to add any new or modified files to the index.

This command can be performed multiple times before a commit. It only adds the content of the specified file(s) at the time the add command is run; if you want subsequent changes included in the next commit, then you must run git add again to add the new content to the index.

The git status command can be used to obtain a summary of which files have changes that are staged for the next commit.

The git add command will not add ignored files by default. If any ignored files were explicitly specified on the command line, git add will fail with a list of ignored files. Ignored files reached by directory recursion or filename globbing performed by Git (quote your globs before the shell) will be silently ignored. The git add command can be used to add ignored files with the -f (force) option.

Please see git-commit(1) for alternative ways to add content to a commit.

How do I use it?

  • Adds content from all *.txt files under Documentation directory and its subdirectories:
    $ git add Documentation/\*.txt 

    Note that the asterisk * is quoted from the shell in this example; this lets the command include the files from subdirectories of Documentation/ directory. 

  • Considers adding content from all git-*.sh scripts:
    $ git add git-*.sh 

    Because this example lets the shell expand the asterisk (i.e. you are listing the files explicitly), it does not consider subdir/git-foo.sh.

    To learn about all the options in details, you can run

     $ man git pull

Git Commit

What is does ?

Records changes to the repository

How  it does ?

The content to be added can be specified in several ways:

  1. by using git add to incrementally “add” changes to the index before using the commit command (Note: even modified files must be “added”);
  2. by using git rm to remove files from the working tree and the index, again before using the commitcommand;
  3. by listing files as arguments to the commit command, in which case the commit will ignore changes staged in the index, and instead record the current content of the listed files (which must already be known to Git);
  4. by using the -a switch with the commit command to automatically “add” changes from all known files (i.e. all files that are already listed in the index) and to automatically “rm” files in the index that have been removed from the working tree, and then perform the actual commit;
  5. by using the –interactive or –patch switches with the commit command to decide one by one which files or hunks should be part of the commit, before finalizing the operation. See the “Interactive Mode” section of git-add(1) to learn how to operate these modes.

The --dry-run option can be used to obtain a summary of what is included by any of the above for the next commit by giving the same set of parameters (options and paths).

If you make a commit and then find a mistake immediately after that, you can recover from it with git reset.

How do I use it?

When recording your own work, the contents of modified files in your working tree are temporarily stored to a staging area called the “index” with git add. A file can be reverted back, only in the index but not in the working tree, to that of the last commit with git reset HEAD -- <file>, which effectively reverts git add and prevents the changes to this file from participating in the next commit. After building the state to be committed incrementally with these commands, git commit (without any pathname parameter) is used to record what has been staged so far. This is the most basic form of the command. An example:

$ edit hello.c
$ git rm goodbye.c
$ git add hello.c
$ git commit

Instead of staging files after each individual change, you can tell git commit to notice the changes to the files whose contents are tracked in your working tree and do corresponding git add and git rm for you. That is, this example does the same as the earlier example if there is no other change in your working tree:

$ edit hello.c
$ rm goodbye.c
$ git commit -a

The command git commit -a first looks at your working tree, notices that you have modified hello.c and removed goodbye.c, and performs necessary git add and git rm for you.

After staging changes to many files, you can alter the order the changes are recorded in, by giving pathnames to git commit. When pathnames are given, the command makes a commit that only records the changes made to the named paths:

$ edit hello.c hello.h
$ git add hello.c hello.h
$ edit Makefile
$ git commit Makefile

This makes a commit that records the modification to Makefile. The changes staged for hello.c andhello.h are not included in the resulting commit. However, their changes are not lost — they are still staged and merely held back. After the above sequence, if you do:

$ git commit

this second commit would record the changes to hello.c and hello.h as expected.

After a merge (initiated by git merge or git pull) stops because of conflicts, cleanly merged paths are already staged to be committed for you, and paths that conflicted are left in unmerged state. You would have to first check which paths are conflicting with git status and after fixing them manually in your working tree, you would stage the result as usual with git add:

$ git status | grep unmerged
unmerged: hello.c
$ edit hello.c
$ git add hello.c

After resolving conflicts and staging the result, git ls-files -u would stop mentioning the conflicted path. When you are done, run git commit to finally record the merge:

$ git commit

Git push

What is does ?

Updates remote refs along with associated objects 

How  it does ?

Updates remote refs using local refs, while sending objects necessary to complete the given refs.

You can make interesting things happen to a repository every time you push into it, by setting up hooksthere. See documentation for git-receive-pack(1).

When the command line does not specify where to push with the <repository> argument,branch.*.remote configuration for the current branch is consulted to determine where to push. If the configuration is missing, it defaults to origin.

When the command line does not specify what to push with <refspec>... arguments or --all--mirror--tags options, the command finds the default <refspec> by consulting remote.*.pushconfiguration, and if it is not found, honors push.default configuration to decide what to push (See git1for the meaning of push.default).

How do I use it?

git push

Works like git push <remote>, where <remote> is the current branch’s remote (or origin, if no remote is configured for the current branch).

git push origin

Without additional configuration, works like git push origin :.

The default behavior of this command when no <refspec> is given can be configured by setting thepush option of the remote, or the push.default configuration variable.

For example, to default to pushing only the current branch to origin use git config remote.origin.push HEAD. Any valid <refspec> (like the ones in the examples below) can be configured as the default for git push origin.

git push origin :

Push “matching” branches to origin. See <refspec> in the OPTIONS section above for a description of “matching” branches.

git push origin master

Find a ref that matches master in the source repository (most likely, it would findrefs/heads/master), and update the same ref (e.g. refs/heads/master) in origin repository with it. If master did not exist remotely, it would be created.

git push origin HEAD

A handy way to push the current branch to the same name on the remote.

git push mothership master:satellite/master dev:satellite/dev

Use the source ref that matches master (e.g. refs/heads/master) to update the ref that matchessatellite/master (most probably refs/remotes/satellite/master) in the mothershiprepository; do the same for dev and satellite/dev.

This is to emulate git fetch run on the mothership using git push that is run in the opposite direction in order to integrate the work done on satellite, and is often necessary when you can only make connection in one way (i.e. satellite can ssh into mothership but mothership cannot initiate connection to satellite because the latter is behind a firewall or does not run sshd).

After running this git push on the satellite machine, you would ssh into the mothership and rungit merge there to complete the emulation of git pull that were run on mothership to pull changes made on satellite.

git push origin HEAD:master

Push the current branch to the remote ref matching master in the origin repository. This form is convenient to push the current branch without thinking about its local name.

git push origin master:refs/heads/experimental

Create the branch experimental in the origin repository by copying the current master branch. This form is only needed to create a new branch or tag in the remote repository when the local name and the remote name are different; otherwise, the ref name on its own will work.

git push origin :experimental

Find a ref that matches experimental in the origin repository (e.g. refs/heads/experimental), and delete it.

Source Code Management — Git (Basic Concepts)

Author : Rahul Krishna Upadhyaya

Topic : Source Code Management

Date : 16/04/2013

Overview : What is source control management? Why do we need to manage the source code? Git as an example of Source Control Management. Some basic commands to get started with working with Git.

Source Control Management.

Source Control Management, as the name would suggest,  is the means by which the source-code for any project is managed. Some of the feature that every SCM-application should contain :

  • It should maintain a incremental history of each file . Every time you tell the SCM-tool that you want to save (commit) the code, it tracks the changes and saves those delta changes in its history.
  • Means to tracks the author of each particular change to every given file being monitored.
  • Provides means to several developers to collaborate and contribute,even while writing to a single file simultaneously, ensuring the integrity of the previously saved code and avoiding adding any new-code which conflicts with the previous version.

Where is the need Source Control Management ?

Q. Case where I am the single Author of the complete content : If I find that I added some code and everything stopped working, and I want to revert back to a state where I know everything was working fine.

A. Every single time you save (commit) your code a commit-id is generated for that. Also, you add a commit message stating some logical reasoning for the changes that you have added. This enables you to go to the history. Choose any commit-id and see all the additions and deletions done in that particular commit and the logical reasoning behind making those changes as the commit message. Here,using this, you can navigate to any particular stage of the  code back in time and check what changes were made or revert back to any stable code-base.

Q. Case where several Authors are writing into the same project and possibly in the same file too.

A. Any source-control management tool would make life really easy for collaboration between a group of developers. You write the changes and “push” this code to a remote central server. All the other developers can then “pull” the changes from this central remote server and update/synchronize their work-spaces with this changed code. If you are working on the same file and making changes to the same section, you will face “conflicts” . You either need to resolve them manually or set a policy to choose theirs or our version of code for updating your work-space with the latest code.

Git – The Most Powerful SCM Tool.

Git is a distributed revision control and source code management (SCM) system with an emphasis on speed.Initially designed and developed by Linus Torvalds for Linux kernel development, Git has since been adopted by many other projects.

Every Git working directory is a full-fledged repository with complete history and full revision tracking capabilities, not dependent on network access or a central server.

A Short History of Git

As with many great things in life, Git began with a bit of creative destruction and fiery controversy. The Linux kernel is an open source software project of fairly large scope. For most of the lifetime of the Linux kernel maintenance (1991–2002), changes to the software were passed around as patches and archived files. In 2002, the Linux kernel project began using a proprietary DVCS system called BitKeeper.

In 2005, the relationship between the community that developed the Linux kernel and the commercial company that developed BitKeeper broke down, and the tool’s free-of-charge status was revoked. This prompted the Linux development community (and in particular Linus Torvalds, the creator of Linux) to develop their own tool based on some of the lessons they learned while using BitKeeper. Some of the goals of the new system were as follows:

  • Speed
  • Simple design
  • Strong support for non-linear development (thousands of parallel branches)
  • Fully distributed
  • Able to handle large projects like the Linux kernel efficiently (speed and data size)

Since its birth in 2005, Git has evolved and matured to be easy to use and yet retain these initial qualities. It’s incredibly fast, it’s very efficient with large projects, and it has an incredible branching system for non-linear development

The Three States- In Local Operations

Now, pay attention. This is the main thing to remember about Git if you want the rest of your learning process to go smoothly. Git has three main states that your files can reside in: committed, modified, and staged. Committed means that the data is safely stored in your local database. Modified means that you have changed the file but have not committed it to your database yet. Staged means that you have marked a modified file in its current version to go into your next commit snapshot.

This leads us to the three main sections of a Git project: the Git directory, the working directory, and the staging area.

Local Operations in Git

Local Operations in Git

The Git directory is where Git stores the metadata and object database for your project. This is the most important part of Git, and it is what is copied when you clone a repository from another computer.

The working directory is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.

The staging area is a simple file, generally contained in your Git directory, that stores information about what will go into your next commit. It’s sometimes referred to as the index, but it’s becoming standard to refer to it as the staging area.

The basic Git workflow goes something like this:

  1. You modify files in your working directory.
  2. You stage the files, adding snapshots of them to your staging area.
  3. You do a commit, which takes the files as they are in the staging area and stores that snapshot permanently to your Git directory.

If a particular version of a file is in the git directory, it’s considered committed. If it’s modified but has been added to the staging area, it is staged. And if it was changed since it was checked out but has not been staged, it is modified.

Git Interaction with Remote Servers

Once as explained above, we are done with the changes and commits in our local system and we want to add these changes to the Remote Git Server – So that the central repository containing the code gets updated and all the peers can synchronize their work-spaces with our changes.

We use git push <some added params> command to push those changes to the remote repositories. At the same time we should know which is the remote repository that we are pointing to : git remote -v  would list down the remote repositories that my current git repository knows of. If there are none , perhaps you need to add one before running a git push command.

A good Step by Step Reference

A Website which you could follow for a step by step comprehensive Learning of git.

Git Immersion

Working on github is also majorly same as working with git on a remote/local machine, only that it has a few more additional things that you need to know.

A good place to learn that would be TryGit Tutorial.

References :