Difference between revisions of "Intro to RNA-Seq Data Analysis Course"

From wiki
Jump to: navigation, search
Line 35: Line 35:
 
Note: If you don't have your password, please ask to have it reset for you.
 
Note: If you don't have your password, please ask to have it reset for you.
  
= Computing resources =
+
= Computing resources notes =
  
 
* RNA-Seq like other Next Generation Sequencing technologies, is characterised by;
 
* RNA-Seq like other Next Generation Sequencing technologies, is characterised by;
Line 52: Line 52:
 
:- We shall use the '''GNU Screen''' utility so we can do other things while waiting.
 
:- We shall use the '''GNU Screen''' utility so we can do other things while waiting.
  
= Aspect of using Windows terminals to connect to Linux =
+
= Computing resources diagram =
 +
 
 
[[File:marv.png]]
 
[[File:marv.png]]
 +
 +
= Aspects of using Windows terminals to connect to Linux =
  
 
* You can run the Windows web-browser (Chrome preferred) and copy text selecting and <code>ctrl+c</code>
 
* You can run the Windows web-browser (Chrome preferred) and copy text selecting and <code>ctrl+c</code>
Line 61: Line 64:
 
:- try to also use tab-completion, and the history (<code>up</code>/<code>down</code> arrows and <code>Ctrl+r</code>)
 
:- try to also use tab-completion, and the history (<code>up</code>/<code>down</code> arrows and <code>Ctrl+r</code>)
  
Weakness to watch out for:
+
<ins>Weakness to watch out for</ins>:
 
* The marvin cluster (more precisely, the network it's attached to) doesn't carry graphics so well.
 
* The marvin cluster (more precisely, the network it's attached to) doesn't carry graphics so well.
 
* We shall be using several graphical programs, and they are all likely to run slowly.
 
* We shall be using several graphical programs, and they are all likely to run slowly.
Line 74: Line 77:
 
* <code>screen</code> works on the ''activator'' key concept, you need to use <code>Ctrl+l</code> (while <code>Ctrl</code>-key iis held down briefly, <code>l</code>-key is pressed) to activate any of its functions.
 
* <code>screen</code> works on the ''activator'' key concept, you need to use <code>Ctrl+l</code> (while <code>Ctrl</code>-key iis held down briefly, <code>l</code>-key is pressed) to activate any of its functions.
 
* After pressing <code>Ctrl+l</code> and releasing you then have a series of single key strokes that will do various useful things.
 
* After pressing <code>Ctrl+l</code> and releasing you then have a series of single key strokes that will do various useful things.
* There will be two command-line windows open when you start it.
+
* There will be one command-line session open when you start it.
 +
:- it's numbered 0, and called <code>scr1</code>, we'll mostly deal with it as <code>0</code>.
 
* Let's learn how to get out of it first
 
* Let's learn how to get out of it first
:- type <code>exit</code>, you should see you have one command-line session less.
 
 
:- type <code>exit</code> again and you will be told you have exited <code>screen</code>.
 
:- type <code>exit</code> again and you will be told you have exited <code>screen</code>.
 
:- you are now back in the ordinary command-line.
 
:- you are now back in the ordinary command-line.
  
 
= GNU Screen 2 =
 
= GNU Screen 2 =
* Go back into screen
+
* Go back into screen, type <code>screen</code>.
* Switch back and forth between the two open sessions: use <code>Ctrl+l,n</code> ('''n''' for next) or <code>Ctrl+l,p</code> ('''p''' for previous)
+
* you open a new session with <code>ctrl+l,c</code> which '''c'''reates a new session.
 +
:- you now have two sessions open
 +
* type <code>ctrl+l,c</code> again, for three open sessions. You can have more, but we'll stick to three: number <code>0</code>, <code>1</code> and <code>2</code>.
 +
* Switch back and forth between the three open sessions: use <code>Ctrl+l,n</code> ('''n''' for next) or <code>Ctrl+l,p</code> ('''p''' for previous)
 
* Don't see anything different when you do this? Look again at the bottom line, the asterisk has changed position.
 
* Don't see anything different when you do this? Look again at the bottom line, the asterisk has changed position.
 
:- the asterisk defines the active session
 
:- the asterisk defines the active session
:- you can move to a numbered session with <code>ctrl+l,1</code> or <code>ctrl+l,2</code> for session no.1 and no.2 respectively.
+
:- you can move to a numbered session directly with <code>ctrl+l,1</code> or <code>ctrl+0,1,2</code> for sessions 0, 1 and 2.
:- you open a new session with <code>ctrl+l,c</code> which '''c'''reates a new session.
+
* you can detech a session, it will stay running, and then go back to it
 +
:- <code>ctrl+l,d</code> to detach
 +
:- Not you're outside screen, type <code>screen -r</code> to reattach.
  
 
= Getting a Queue slot =
 
= Getting a Queue slot =
Line 94: Line 102:
 
* Type <code>qrsh</code> which requests a queue slot ... it will take a little time to give you one.
 
* Type <code>qrsh</code> which requests a queue slot ... it will take a little time to give you one.
 
:- we shall not use this slot for the graphical programs, only the processing ones.
 
:- we shall not use this slot for the graphical programs, only the processing ones.
:-  
+
:-
 
 
= Overview of RNA-Seq =
 
 
 
* For gene expression analyses, seen as a more powerful replacememnt to microarrays
 

Revision as of 15:14, 11 May 2017

Course schedule

  • This is based on a 2 day Edinburgh Genomics course of the same name, with the following changes:
- "Introduction to Linux" moduel excluded
- "Sequencer technology overview" module excluded.
- No laboratory visit
- 50% of that course was theoretical, this will be reduced to 30%
  • Each section begins with a "Talk", and then a practical runthrough.
  • If necessary, some talk slides may be skipped, as the main idea is getting through the practicals.
  • Having said that if major theoretical points arise during a practical, they will be discussed.
  • Course website: http://stab.st-andrews.ac.uk/i2rda/
- this has all the presentations and practicals

Connecting to a remote Machine

Presenting this before introduction as some people might experience delays logging in.

  • We shall use a remote machine not the machine you are logged into locally
  • The program we shall use is PuTTY.
  • Please try to locate PuTTY in the applications section or on AppsAnywhere

Configuring PuTTY for connection

  • Server: marvin.st-andrews.ac.uk
  • Terminal | keyboard | check VT100+
  • Window | Selection | Control use of Mouse | set xterm
  • Connect | Data | enter username
  • Connection | ssh | X11 Forwarding | Check yes
  • Back to PuTTY main screen | select Default setting |click save

You should now be able to "open" a session

  • Be aware: typing in your password is done blindly. I.e. it does not appear on the screen.

Note: If you don't have your password, please ask to have it reset for you.

Computing resources notes

  • RNA-Seq like other Next Generation Sequencing technologies, is characterised by;
- heavy computational workloads
- many different software programs, sometimes doing the same thing, which can be arranged into a pipeline.
- long-running tasks.

These have three implications:

- The marvin cluster is an 11-machine shared computing resource, not a personal computer ... others are using it.
- We need to load the special software before using it
- We want to be able to have a process run unattended.
  • For these three aspects, we have:
- A queue system to use, we shall request an interactive session (qrsh) from the queue.
- Use the module system to load, list and unload software programs
- We shall use the GNU Screen utility so we can do other things while waiting.

Computing resources diagram

Marv.png

Aspects of using Windows terminals to connect to Linux

  • You can run the Windows web-browser (Chrome preferred) and copy text selecting and ctrl+c
  • This can then be pasted inside the putty command-line by clicking the middle mouse button.
  • In many ways, copy-pasting is not great for learning.
- although some of the commands are too long to type out, even with history and tab-completion.
- try to also use tab-completion, and the history (up/down arrows and Ctrl+r)

Weakness to watch out for:

  • The marvin cluster (more precisely, the network it's attached to) doesn't carry graphics so well.
  • We shall be using several graphical programs, and they are all likely to run slowly.
- and sometimes even stall
- we'll cross that bridge when we come to it.

GNU Screen 1

A program which allows several command-line sessions open, similar to the idea of open tabs in a web browser. Let's try it out.

  • To enter a new session, type screen
  • This will open with quite a bare screen except with a indicator line at the bottom.
  • screen works on the activator key concept, you need to use Ctrl+l (while Ctrl-key iis held down briefly, l-key is pressed) to activate any of its functions.
  • After pressing Ctrl+l and releasing you then have a series of single key strokes that will do various useful things.
  • There will be one command-line session open when you start it.
- it's numbered 0, and called scr1, we'll mostly deal with it as 0.
  • Let's learn how to get out of it first
- type exit again and you will be told you have exited screen.
- you are now back in the ordinary command-line.

GNU Screen 2

  • Go back into screen, type screen.
  • you open a new session with ctrl+l,c which creates a new session.
- you now have two sessions open
  • type ctrl+l,c again, for three open sessions. You can have more, but we'll stick to three: number 0, 1 and 2.
  • Switch back and forth between the three open sessions: use Ctrl+l,n (n for next) or Ctrl+l,p (p for previous)
  • Don't see anything different when you do this? Look again at the bottom line, the asterisk has changed position.
- the asterisk defines the active session
- you can move to a numbered session directly with ctrl+l,1 or ctrl+0,1,2 for sessions 0, 1 and 2.
  • you can detech a session, it will stay running, and then go back to it
- ctrl+l,d to detach
- Not you're outside screen, type screen -r to reattach.

Getting a Queue slot

We're going to use one of the screen sessions to get a slot from the queue.

  • Assuming you've launched screen, type ctrl+l,0 to confirm you are in the first screen session.
  • Type qrsh which requests a queue slot ... it will take a little time to give you one.
- we shall not use this slot for the graphical programs, only the processing ones.
-