Why a Queue Manager?

Introduction

With so many personal devices around nowadays: laptops, smartphones, etc., it's normal to ask the question, "why must I submit to a queue manager?".

Most of this can be explained by the personal nature of such devices, and how this can lure users into thinking that queuing is really a bad dream from the past which is happily becoming rarer now.

However, a look around modern life will show that perhaps the activity of queuing takes less time now (especially if one compares to periods of extreme shortages of staples such as bread and gasoline) but that there are in fact more queues now.

What has occurred to decrease the time spent queuing is that queue management is taken much more seriously now than in the past. Many techniques have now been put in place to make queuing both shorter and less stressful. There has been a growth in centralisation and sharing of services, but modern technologies have made queuing less of burden, and, at time, undetectable.

This is the case with marvin computer cluster, it is a shared resource which has a very efficient queue manager called Grid Engine, which will be explained here. Yes, it does mean some small inconveniences, but if one undertands that it is shared by a significant number of people working on different research projects

Why?

computational intensity

Scientific research projects tend to attract some very compute-intensive workloads. These are not, in fact, so common in the biological sciences (espceially when compared to physics and chemistry), but nevertheless, research projects are often trying to solve some thorny problems, and some programs may resort to trying out all possible solutions and seeing which gives the right answer. Because the machines are on 24 hours a day, and users also do not have to supervise the processing (they can go away and come back later), some workloads are capable of taking up all possible resources from other programs.

This is not a case of bad manners, because it is natural, it will tend to occur. So this is why using a queue manager is a good thing to consider as routine.