GalaxyDevelopment – galaxy

Context Navigation

close Warning: Can't synchronize with repository "(default)" (Unsupported version control system "git": Can't find an appropriate component, maybe the corresponding plugin was not enabled? ). Look in the Trac log for more information.

TOC?

Directory and repository setup (one time)

all-in-one script

Automated script for running the below described process: setup_personal_galaxy.sh:

# checkout helper scripts
cd ~
git clone https://github.com/rusalkaguy/uab_ngs.git
# create a new personal galaxy from scratch
~/uab_ngs/galaxy/setup_personal_galaxy.sh

Detailed manual protocol

Create galaxy project directory in your home directory as shown below. Following directory structure is used by galaxy/galaxy-developer module, so you should keep it the same if you want to use that module. Alternatively, you can create your own module file and hide your projects/directory structure. This is described in the next bullet point.
```
$ mkdir -p $HOME/projects/galaxy
$ cd $HOME/projects/galaxy 
```

You can create your own galaxy-developer module file as described below.

Create .modulefiles directory if necessary and place following galaxy-developer module file in it. Following instructions assume file name as 'galaxy-developer', but you can use any other file name as well.

$ mkdir -p ~/.modulefiles

#%Module

# galaxy module file
# does galaxy even have a version?
set galaxyver 1.0
set home $::env(HOME)
# galaxyinstall - personal install root - CHANGE following line to point to your galaxy clone 
set galaxyinstall $home/projects/galaxy/galaxy
# galaxyroot - core/main galaxy root, personal installs will use it's Python and galaxy-tools
set galaxyroot /share/apps/galaxy
set url "https://galaxy.uabgrid.uab.edu/"
set msg "This module adds Galaxy v$galaxyver to various paths\n\nPython Official Site: $url\n"


proc ModulesHelp { } {
   puts stderr "$msg"
}

module-whatis   "$msg"

setenv GALAXY_TOOLS $galaxyroot/galaxy-tools
setenv GALAXY_PYTHON $galaxyroot/python
# Use personal install galaxy library files and not main galaxy library files 
setenv GALAXY_LIB $galaxyinstall/lib

prepend-path PATH $galaxyroot/galaxy-tools/bin
prepend-path LD_LIBRARY_PATH $galaxyroot/galaxy-tools/lib:$galaxyroot/galaxy-tools/lib64:$galaxyroot/galaxy-tools/lib64/R
# pythonpath should search for library files in the personal install
setenv PYTHONPATH $galaxyinstall/lib

# load galaxy sub-modules
module load galaxy/galaxy-submodule-python-2.6
module load galaxy/galaxy-submodule-perl-5.18
module load galaxy/galaxy-submodule-gnuplot-font
module load galaxy/galaxy-submodule-drmaa
module load galaxy/galaxy-submodule-java
module load galaxy/galaxy-submodule-mercurial
module load galaxy/galaxy-submodule-MACS
module load galaxy/galaxy-submodule-weblogo
module load galaxy/galaxy-submodule-meme

Change 'set galaxyinstall <location>' line in above file to your galaxy cloned repository. For example, if you are cloning repository directly in home directory then your galaxyinstall will be '$home/galaxy'.

Load galaxy code developer module:

# See list of available modules first 
$ module avail

# Load this if you are using generic module file with galaxy code in $HOME/projects/galaxy/galaxy 
$ module load galaxy/galaxy-developer

# Load this if you created a module file in ~/.modulefiles directory. Change module file name if necessary. 
$ module load galaxy-developer

Clone galaxy repository from 'official' galaxy-uab repository

$ cd $HOME/projects/galaxy/
# ${USER} should be your BlazerID!
$ git clone ssh://${USER}@git.uabgrid.uab.edu/home/git/repositories/galaxy.git

Checkout develop branch: We will be using galaxy develop branch in our personal as well as deployed install. We can change this convention later, but that's the convention/workflow followed right now.
```
$ cd $HOME/projects/galaxy/galaxy
$ git branch --track develop origin/develop
$ git checkout develop
# can also do the above in a single step
$ git checkout -b develop origin/develop
```

Run it!

Before running add the following line to run.sh
```
unset -f module
```
Configure your instance by running 'personalize.sh' script. The script will change galaxy server port number and file/directory paths. It will not change any user authentication or administration parameters. You will need to modify these parameters manually for now.
```
$ ./personalize.sh 
```

Start galaxy by running run.sh process. I recommend starting it with '--daemon' option. Note Python Paste server has some bug with Python 2.6 which prevents killing non-daemon server process using Ctrl+C.
```
$ ./run.sh --daemon
```

Stop galaxy daemon:

$ ./run.sh --stop-daemon 
$ ./run.sh --help # See all available options

Accessing galaxy web interface using port forwarding: The galaxy server will be listening only on localhost, hence you will need to access it using ssh port forwarding. Following ssh command will forward your local system's port to a port on the Cheaha where galaxy service is running. Run this command from your local system and not Cheaha.
```
$ ssh -L <local_port>:127.0.0.1:<galaxy_port> <blazerid>@cheaha.uabgrid.uab.edu 
# Now you can access galaxy by typing - http://localhost:<local_port> in browser 
```
Windows users will need to set up a tunnel through PuTTy.

Develop it!

The distributed (I prefer calling it 'decentralized') repository nature of Git allows numerous development workflows. In case of centralized repositories like Subversion one has to commit changes to the central Subversion server in order to record/version-control changes. However, the decentralized repositories like Git allow you to record your changes locally and share them with other developers. This facilitates some useful workflows as explained graphically here. We are using integration-manager workflow for our galaxy development and following are suggested steps for doing development.

Making and sharing changes

Update develop branch

$ git checkout develop
$ git pull

Create a ticket for your change set

This is optional if you are making changes in your local environment that won't affect others.

Create a new branch for your ticket/changeset

You should always make changes in a non-develop branch unless you are repository integrator and/or you know how to rollback/revert your changes in a clean manner using advanced Git commands like rebase, reset etc. I highly recommend keeping your develop branch read-only and working on a separate issue specific branch. If possible create a branch using ticket number so that it will be convenient (self-describing) while referencing/sharing it with others.

$ git checkout -b <describe-bugfix-branch>

Commit changes to your ticket/changeset

I have not covered (advanced) Git commands that let you add commits interactively or change previous commit contents. But such commands are really useful and common with many git based development. I highly recommend getting familiar with these commands and using it in your development workflow (may add more details on it later). Also, please mention ticket number using trac wiki formatting language in the commit message as a cross-reference. See ProGit's Contributing to a Project chapter for commit guidelines and other useful information on project workflows/contribution.

# git add-commit 
$ git add <filename>
$ git commit -m "Fix for [ticket:12]"

Test changes locally

Sharing you changes

You can share your changes in two ways:

Share the branch with other developers
- For this you will need to modify file system permissions so that other people can access your code directory/repository. Then send access details - repository path and branch name - with other developers.

Create a patch for changesets and share it with other users

# Following command will create a patch file(s) for commits between commit1 and commit2 (displayed after doing a git merge)
$ git format-patch <commit1>..<commit2>

Email this patch file to galaxy-dev@… and/or pavgi@…

# Email command example 
$ /usr/sbin/sendmail galaxy-dev@vo.uabgrid.uab.edu < 0001-*.patch

Update the ticket information and attach patch file(s).

Incorporating / Merging other user's changes

Will be modified soon - Not wrong, but needs to be clarified

Getting / Pulling changes from develop

Pull develop from the UAB master copy, then rebase your working branch

# pull from master
git checkout develop
git pull

# update my working branch
git checkout ticket_00_branch
git rebase develop

Getting / Pulling changes from other developers

Add remote for the other developer's git repository.

# Add remote for a developer's repository  (-f option performs fetch after remote is added)
$ git remote add -f <blazerid-developer> <repository-URL>  

# Verifying changes/diff against developer's branch 
$ git log -p develop..remotes/<blazerid-developer>/<developer's branch>

# Create a local tracking branch for developer's repo as follows
$ git checkout -b blazerid-develop <blazerid-developer>/develop

Merging changes obtained from other developers

# Merge developer's branch changes with your 'blessed' develop branch 
$ git checkout develop
# Merge from tracking branch 
$ git merge <blazerid-developer>-develop 
# Alternatively you can merge directly using local ref to remote branch 
$ git merge remotes/<blazerid-developer>/<developer's branch>

# Push changes to the blessed repository 
$ git checkout develop 
$ git push

Reference

http://whygitisbetterthanx.com/#any-workflow
https://code.djangoproject.com/wiki/CollaborateOnGithub - Explains integration manager workflow with git commands

Last modified 9 years ago Last modified on Jul 25, 2014, 8:20:02 PM

Download in other formats:

Plain Text