Difference between revisions of "StABDMIN"
(→Restarting the server) |
PeterThorpe (talk | contribs) (→StABDMIN Site) |
||
| (5 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
=StABDMIN Site= | =StABDMIN Site= | ||
| − | Currently running at: marvin.st-andrews.ac.uk on port 80. | + | Currently running at: |
| + | marvin.st-andrews.ac.uk on port 80. | ||
| + | |||
| + | thi sis the stabmin / admin / tracking website | ||
The StABDMIN site was born of the need of a way to track the users on marvin, the PIs they're associated with, the funded and unfunded grants they hold and the data they use. | The StABDMIN site was born of the need of a way to track the users on marvin, the PIs they're associated with, the funded and unfunded grants they hold and the data they use. | ||
| − | It was written by Joe in about 2 weeks in 2019 and he apologises sincerely for the lack of comments in the code, and absence of tests. It's still a damn site better than what was there before (i.e. nothing). | + | It was written by Joe in about 2 weeks in 2019 and he apologises sincerely for the lack of comments in the code, and absence of tests. It's still a damn site better than what was there before (i.e. nothing). |
=Keeping it running= | =Keeping it running= | ||
| Line 19: | Line 22: | ||
As root: | As root: | ||
| − | *To start the webserver navigate to /storage/home/users/StABDMIN/StABDMIN | + | *To start the webserver navigate to |
| − | * run | + | /storage/home/users/StABDMIN/StABDMIN |
| − | * run | + | * run |
| + | module load python/3.6.4 | ||
| + | * run | ||
| + | sh run_webserver.sh | ||
And this will run gunicorn using | And this will run gunicorn using | ||
| Line 45: | Line 51: | ||
=How it works= | =How it works= | ||
| − | The whole thing is based on the python package Django, but uses javascript for loading data into datatables (javascript package that makes the tables pretty), using JQuery (js again) ajax calls. The plots are created by D3 (javascript plotting library), again from data from ajax calls. | + | The whole thing is based on the python package Django, but uses javascript for loading data into datatables (javascript package that makes the tables pretty. All javascript is in the template html file for the page in templates/), using JQuery (js again) ajax calls. The plots are created by D3 (javascript plotting library), again from data from ajax calls. |
There are 5 tables in the database directly used by django. | There are 5 tables in the database directly used by django. | ||
| Line 54: | Line 60: | ||
help notes stores notes on help given, meaning we can track what we've done for people. These can be assigned to users, PIs and/or grants. | help notes stores notes on help given, meaning we can track what we've done for people. These can be assigned to users, PIs and/or grants. | ||
Bioinformaticians is a way of highlighting user instances as bioinformaticians so we can assign them to grants. | Bioinformaticians is a way of highlighting user instances as bioinformaticians so we can assign them to grants. | ||
| + | |||
| + | |||
| + | ====Files and Folders==== | ||
| + | *accounts - has the code of the login side of the site | ||
| + | *cronTask.sh - attempt to get cron to load the data | ||
| + | *importUsageFromFiles.py - script to load the data in from /mnt/system-usage/ | ||
| + | *manage.py - core django management script. | ||
| + | *mysql - password for mysql database | ||
| + | *run_webserver.sh - script that'll run a daemonized gunicorn webserver for running the website. if it's down, run this. | ||
| + | *StADBMIN - core of the site is here, settings.py and urls.py most useful | ||
| + | *StADBDMIN.pid - process id of gunicorn webserver running the site | ||
| + | *static - static files, populate using "python3 manage.py collectstatic" to get the static files from the apps together | ||
| + | *templates - template html files the site servers for the main views. The javascript for the plots are in here. | ||
| + | *UsersGrants - the core code for the site. All of the functions that actually run the site are in view.py, the urls are managed in urls.py | ||
| + | *usersImport.yaml - this was the initial import of data to the database. likely outdated structure already due to changes, but kept incase the database is lost | ||
| + | *wsgi.py - manages the wsgi stuff. I think this isn't needed. | ||
Latest revision as of 08:53, 3 March 2020
Contents
StABDMIN Site
Currently running at:
marvin.st-andrews.ac.uk on port 80.
thi sis the stabmin / admin / tracking website
The StABDMIN site was born of the need of a way to track the users on marvin, the PIs they're associated with, the funded and unfunded grants they hold and the data they use.
It was written by Joe in about 2 weeks in 2019 and he apologises sincerely for the lack of comments in the code, and absence of tests. It's still a damn site better than what was there before (i.e. nothing).
Keeping it running
The database is called StABDMIN, the database user is StABDMIN and the password is stored in /etc/mysql/stabdmin.cnf.
The code for the project is in /storage/home/users/StABDMIN. The site is set to store static files (css, js etc) in /var/www/StABDMIN/.
Currently static files are being server using the python package "whitenoise", but this ought to be done with nginx or apache in the long run. It's also not served over https currently as marvin can't cope with a modern apache (mod_wsgi was compiled with python 2.6).
Restarting the server
As root:
- To start the webserver navigate to
/storage/home/users/StABDMIN/StABDMIN
- run
module load python/3.6.4
- run
sh run_webserver.sh
And this will run gunicorn using
gunicorn StABDMIN.wsgi:application --pid StABDMIN.pid -b marvin.st-andrews.ac.uk:80 -n StABDMIN -D
You can stop it with the process id in StABDMIN.pid file, and/or by using pkill gunicorn
Importing data into the backend
The script /storage/home/users/StABDMIN/StABDMIN/importUsageFromFiles.py is set into cron to run everyday at 5.05am. It relies on the result of the /mnt/system_usage/ scripts written by Peter Thorpe. It loads the data from the files created by this script into the system for tracking user data usage and total disk use/free space.
module load python/3.6.4 python3 importUsageFromFiles.py
will load the data in manually.
To add users/PIs/Grants, you need to use the ADMIN site, access from the "Admin" link on the navigation bar of the website.
If a new PI has a new Users and a new Grant, the PI must be created first.
If a new bioinformatician joins, assign them the PI "StABU" and add them to the bioinformatician table. This allows tracking of their use within the StABU PI, and also allows them to be assigned as primary bioinformaticians on grants.
How it works
The whole thing is based on the python package Django, but uses javascript for loading data into datatables (javascript package that makes the tables pretty. All javascript is in the template html file for the page in templates/), using JQuery (js again) ajax calls. The plots are created by D3 (javascript plotting library), again from data from ajax calls.
There are 5 tables in the database directly used by django.
Pis stores all the PI information. GrantSubmissions stores the grants information, whether it's proposals, expired, or funded. Users stores the information on users. help notes stores notes on help given, meaning we can track what we've done for people. These can be assigned to users, PIs and/or grants. Bioinformaticians is a way of highlighting user instances as bioinformaticians so we can assign them to grants.
Files and Folders
- accounts - has the code of the login side of the site
- cronTask.sh - attempt to get cron to load the data
- importUsageFromFiles.py - script to load the data in from /mnt/system-usage/
- manage.py - core django management script.
- mysql - password for mysql database
- run_webserver.sh - script that'll run a daemonized gunicorn webserver for running the website. if it's down, run this.
- StADBMIN - core of the site is here, settings.py and urls.py most useful
- StADBDMIN.pid - process id of gunicorn webserver running the site
- static - static files, populate using "python3 manage.py collectstatic" to get the static files from the apps together
- templates - template html files the site servers for the main views. The javascript for the plots are in here.
- UsersGrants - the core code for the site. All of the functions that actually run the site are in view.py, the urls are managed in urls.py
- usersImport.yaml - this was the initial import of data to the database. likely outdated structure already due to changes, but kept incase the database is lost
- wsgi.py - manages the wsgi stuff. I think this isn't needed.
