Cluster Slot Usage (CPU & Memory) Report
This table shows cluster cpu/memory usage for qsub jobs in terms of “Slot Equivalent Time” by periods of 'day' and 'division'. qlogin jobs/sessions are not reported.
The goal of this report is to help with deciding what how many slots per “High-speed Project Slot Quota” to request for your group. Click here for an explanation of slots and Project Slot Quotas.
Quick notes on slots and Project Slot Quotas:
- A “slot” is a cpu core and 6GB of RAM (3GB on the basic nodes).
- Each Group/Lab/Billing-Entity has a “Basic Project Slot Quota” as a part of the required Basic Account. See the link above for details. This report is meant for deciding how many slots to choose for the optional “High-speed” Project Slot Quotas.
- A user can be in any number of different Project Slot Quotas, and can be in Project Slot Quotas that belong to different Group/Lab/Billing-Entities.
- Users can have different per-user slots quotas within a Project Slot Quota.
- Slots are added to Project Slot Quotas in increments of 16, so there is a 16-slot minimum for a High-speed Project Slot Quota.
NOTE Project Slot Quotas will be put into use very soon. Once this is done, each job will be assigned to an SGE project (with the same name as a Project Slot Quota) to track and limit resources by Project Slot Quota. For users who work with multiple groups or labs, this means their quotas will then be set based on the project they assign to their jobs, allowing them to work flexibly with multiple labs. However, the report below is from jobs that do not include that information, so users who work within multiple groups have their usage below reported only as a part of their 'home' group/lab.
TERMINOLOGY
# of non-zero days
This column shows how many days of the reporting period had at least one job running. All statistics are computed only over days or divisions (see below) during which one or more jobs were run. If you ran a total of 100 jobs within the reporting period, but they were all on a single day, your daily job average will be 100.
Slot Equivalent
A “Slot Equivalent” (SE) represents a job's fractional SGE quota usage in units of either one cpu core or 6GB of RAM (whichever is greater for a job). We use 6GB because for each slot in a user's quota, 6GB of RAM quota is allotted.
Examples:
| Requested slots/cores | Requested memory | Slot Equivalent |
|---|---|---|
| 1 | 3GB (default) | 1 |
| 1 | 9GB | 1.5 |
| 2 | 9GB | 2 |
| 2 | 20GB | 3.33 (20GB / 6GB) |
Slot Equivalent Time (SET)
The “Slot Equivalent Time” (SET) is the period of time a job runs, multiplied by the SE of the job, reported in units of days or 'divisions'.
SET = SE * job-duration
So a job that runs for 8 hours with 1 SE is reported as running for 0.33 “SET by Day”. A job running for 16 hours with 2 SE is reported using 1.33 “SET by Day” (2 SE * 16 hours / 24 hours-per-day) ). SET is also reported by 'division', a period shorter than a full day (the value of which is listed below). For a division period of 4 hours then, a 6 hour job with 1 SE would be reported as 1.5 “SET by Division” ( 1 SE * 6 hours / 4 hours-per-division).
If you ran 1 job that lasted 24 hours, you'd get an “SET by Day” of 1, and an “SET by Division” of 1. But if you ran 24 1-hour jobs that all ran within the same 4-hour window, you'd still get an “SET by Day” of 1, however the “SET by Division” would be 24. This would show your jobs were more concentrated in a smaller window of time.
The purpose of this metric is to get a useful idea of how many SE units are used at once by a group over useful periods of time, and so have a good idea of what kind of slot-group quota the group needs.
Long jobs
Jobs that last longer than a day or a division are spread out over as many days or divisions they straddle.
How to use this report
The most important values are the first three columns: '# of non-zero days', and the average and standard deviation values reported under 'SET By Day'.
Check how many days your group had jobs running, and then the SET values. For example if your number of non-zero days is high compared to the number of days in the report, you may want to simply add the average “SET by Day” to its standard deviation to allow users to generally run the same number of jobs as they have in the past. If your number of non-zero days is low, consider just the “SET by Day” average to save on Project Slot Quota fees.
You can also consider the “SET By Div” data. If these values are significantly higher, it probably indicates users are running (possibly shorter jobs) during the same time periods, e.g. during the work day. In that case you may want to use these numbers to decide how many slots to request.
== REPORT ==
(note, I will refine the output formatting in future reports)
Period Begin: Sat Jun 4 17:35:37 EDT 2016
Period End : Fri Nov 11 16:35:37 EST 2016
Days: 160
Hours per Division: 4 (for stats reported under “SET By Div.”)
| # of nonzero days (out of 160) | SET By Day | # of Jobs by day | Avg time per job | SET By Div | Jobs by div. | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Group | Avg | Std | Median | Max | Total all days | Avg | Std | Median | Max | Minutes | Avg | Std | Median | Max | Avg | Std | |
| admin | 70.00 | 1.68 | 1.57 | 1.105 | 9.17 | 3756 | 53.66 | 48.30 | 40.000 | 212.00 | 22.522 | 6.78 | 6.48 | 4.354 | 28.34 | 36.15 | 39.91 |
| Aguirre - TOME | 61.00 | 18.62 | 16.48 | 12.554 | 63.15 | 842382 | 13809.54 | 29972.98 | 32.000 | 148103.00 | .795 | 26.01 | 16.97 | 25.413 | 83.66 | 3219.33 | 6910.41 |
| Aguirre - MELA | 71.00 | 21.56 | 22.56 | 14.537 | 111.72 | 817578 | 11515.18 | 27772.44 | 32.000 | 135858.00 | 1.206 | 30.72 | 22.39 | 28.032 | 102.18 | 2741.87 | 6467.17 |
| Ashtari | 15.00 | 7.95 | 5.63 | 5.589 | 16.17 | 514 | 34.27 | 39.40 | 15.000 | 142.00 | 334.067 | 10.37 | 5.54 | 11.000 | 23.28 | 15.52 | 20.57 |
| Avants | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Bassett | 141.00 | 42.20 | 28.52 | 42.480 | 132.76 | 64438 | 457.01 | 1480.13 | 69.000 | 11431.00 | 102.926 | 45.22 | 29.81 | 42.480 | 145.88 | 108.08 | 437.96 |
| Brannon | 40.00 | 14.61 | 18.01 | 3.968 | 49.28 | 15673 | 391.82 | 572.51 | 124.000 | 2763.00 | 51.652 | 27.79 | 20.13 | 37.162 | 87.67 | 136.24 | 267.34 |
| Burdick | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Buxbaum | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| CBIG | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Chatterjee | 44.00 | 8.48 | 15.31 | 1.037 | 49.09 | 9176 | 208.55 | 305.80 | 51.000 | 1076.00 | 17.342 | 19.13 | 20.58 | 6.224 | 58.05 | 78.61 | 100.03 |
| Coslett | 63.00 | 13.82 | 14.76 | 5.699 | 43.41 | 29454 | 467.52 | 1026.17 | 53.000 | 4320.00 | 34.940 | 21.69 | 16.60 | 23.556 | 54.99 | 134.31 | 475.08 |
| Davis | 29.00 | 10.61 | 13.81 | 2.833 | 45.53 | 1120 | 38.62 | 65.00 | 10.000 | 250.00 | 136.389 | 15.79 | 18.91 | 5.312 | 91.07 | 13.40 | 28.76 |
| # of nonzero days (out of 160) | SET By Day | # of Jobs by day | Avg time per job | SET By Div | Jobs by div. | ||||||||||||
| Group | Avg | Std | Median | Max | Total all days | Avg | Std | Median | Max | Minutes | Avg | Std | Median | Max | Avg | Std | |
| Detre | 81.00 | 8.49 | 10.67 | 5.945 | 51.79 | 39845 | 491.91 | 1079.01 | 62.000 | 5198.00 | 13.937 | 13.33 | 14.16 | 6.083 | 69.21 | 134.99 | 476.86 |
| Epstein | 83.00 | 10.75 | 12.66 | 6.832 | 54.24 | 23119 | 278.54 | 576.08 | 95.000 | 3242.00 | 10.614 | 16.13 | 16.78 | 8.416 | 84.40 | 71.80 | 236.20 |
| Farah | 23.00 | 32.82 | 14.00 | 36.457 | 52.60 | 2158 | 93.83 | 195.79 | 55.000 | 966.00 | 437.821 | 36.53 | 12.50 | 40.764 | 67.37 | 41.81 | 38.10 |
| Farrar | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Gee | 76.00 | 12.52 | 15.27 | 6.255 | 75.86 | 4434 | 58.34 | 112.23 | 23.000 | 826.00 | 134.582 | 18.78 | 19.12 | 11.080 | 83.10 | 20.67 | 36.39 |
| Gee_training | 35.00 | 2.00 | 3.17 | .884 | 16.04 | 3567 | 101.91 | 162.54 | 25.000 | 580.00 | 9.452 | 4.74 | 7.62 | 2.017 | 39.42 | 39.17 | 74.00 |
| Grossman | 135.00 | 14.37 | 23.45 | 3.291 | 117.58 | 33955 | 251.52 | 660.35 | 22.000 | 3775.00 | 38.775 | 23.48 | 27.17 | 12.211 | 137.26 | 76.88 | 279.12 |
| Gur | 143.00 | 55.97 | 47.49 | 47.408 | 195.07 | 1326517 | 9276.34 | 14093.56 | 3989.000 | 86969.00 | 6.353 | 66.80 | 51.95 | 55.837 | 676.94 | 1863.18 | 3884.90 |
| Hamilton | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Kable | 40.00 | 7.37 | 12.95 | .951 | 47.48 | 433277 | 10831.92 | 40484.60 | 98.000 | 204907.00 | .912 | 17.60 | 21.44 | 7.353 | 86.10 | 4381.47 | 13273.23 |
| Kim_J | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Kofke | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Lerman | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| # of nonzero days (out of 160) | SET By Day | # of Jobs by day | Avg time per job | SET By Div | Jobs by div. | ||||||||||||
| Group | Avg | Std | Median | Max | Total all days | Avg | Std | Median | Max | Minutes | Avg | Std | Median | Max | Avg | Std | |
| Loughead | 94.00 | 5.96 | 9.48 | 3.518 | 56.26 | 101696 | 1081.87 | 1848.10 | 125.000 | 8861.00 | 5.441 | 11.23 | 14.39 | 4.705 | 62.45 | 343.90 | 750.02 |
| Mackey | 3.00 | 0.34 | 0.01 | .339 | 0.34 | 3 | 1.00 | 0.00 | 1.000 | 1.00 | 487.716 | 0.76 | 0.44 | 1.000 | 1.00 | 1.00 | 0.00 |
| Medaglia | 40.00 | 4.11 | 5.65 | 2.993 | 29.96 | 338 | 8.45 | 13.98 | 4.000 | 60.00 | 609.949 | 4.73 | 7.47 | 3.000 | 47.62 | 5.41 | 8.64 |
| Radiology | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Rao | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Rizi | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Romer | 8.00 | 1.08 | 1.62 | .473 | 4.64 | 55 | 6.88 | 8.13 | 7.000 | 26.00 | 88.810 | 1.68 | 1.89 | 1.000 | 5.46 | 2.48 | 3.63 |
| Schwartz | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| Sheline | 38.00 | 2.24 | 3.67 | .317 | 11.90 | 7500 | 197.37 | 360.51 | 40.000 | 1811.00 | 7.355 | 5.88 | 6.71 | 2.469 | 29.58 | 86.49 | 233.19 |
| Smith | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |
| test1 | 15.00 | 0.07 | 0.09 | .057 | 0.24 | 291 | 19.40 | 32.50 | 6.000 | 121.00 | 2.303 | 0.38 | 0.51 | .048 | 1.47 | 16.17 | 30.34 |
| # of nonzero days (out of 160) | SET By Day | # of Jobs by day | Avg time per job | SET By Div | Jobs by div. | ||||||||||||
| Group | Avg | Std | Median | Max | Total all days | Avg | Std | Median | Max | Minutes | Avg | Std | Median | Max | Avg | Std | |
| Thompson-Schill | 79.00 | 8.13 | 8.30 | 5.885 | 39.72 | 12873 | 162.95 | 327.71 | 7.000 | 1596.00 | 23.031 | 10.76 | 10.69 | 6.666 | 53.98 | 37.69 | 141.78 |
| Wehrli | 56.00 | 3.46 | 3.76 | 1.678 | 13.11 | 5921 | 105.73 | 483.69 | 4.000 | 3349.00 | 17.927 | 5.68 | 5.30 | 4.516 | 33.09 | 30.29 | 186.30 |
| Wolf | 2.00 | 0 | 0 | 0 | 0.00 | 3 | 1.50 | 0.71 | 2.000 | 2.00 | .260 | 0.00 | 0.00 | .001 | 0.00 | 1.00 | 0.00 |
| Wolk | 57.00 | 7.69 | 9.97 | 2.509 | 32.37 | 61630 | 1081.23 | 1361.96 | 681.000 | 6086.00 | 6.604 | 15.47 | 16.78 | 7.388 | 74.40 | 361.68 | 503.34 |
| Yushkevich | 114.00 | 17.70 | 17.61 | 12.146 | 64.72 | 170105 | 1492.15 | 2239.33 | 674.000 | 12099.00 | 12.204 | 26.49 | 19.85 | 27.663 | 94.98 | 373.54 | 636.74 |
