Computing

User Tools

Site Tools

Trace:
test_table

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
test_table [2016/12/07 19:50]
mgstauff 		test_table [2016/12/14 16:21] (current)
mgstauff
Line 1: Line 1:
-**Cluster CPU/Memory Usage Report**+**Cluster Slot Usage (CPU & Memory) Report**
  
 This table shows cluster cpu/memory usage for ''qsub'' jobs in terms of "Slot Equivalent Time" by periods of 'day' and 'division'. ''qlogin'' jobs/sessions are not reported. This table shows cluster cpu/memory usage for ''qsub'' jobs in terms of "Slot Equivalent Time" by periods of 'day' and 'division'. ''qlogin'' jobs/sessions are not reported.
  
-The goal of this report is to help with deciding what how many slots per "High-speed Slot Group" to request for your group. For a discussion on slots and Slot Groups, [[cluster_billing#slot_groups|click here]].+The goal of this report is to help with deciding what how many slots per "High-speed Project Slot Quota" to request for your group. For a discussion on slots and Project Slot Quotas, [[cluster_billing#slot_groups|click here]]. Note that slots are added to Project Slot Quotas in increments of 16, so there is a 16-slot minimum for a High-speed Project Slot Quota.
  
-__NOTE__ Slot Group quotas have not yet been implemented on the cluster, but will be very soon. Once this is done, each job will be assigned to an SGE project (with the same name as a slot group) to track and limit by Slot Group quotas. For users who work with multiple groups or labs, this means their quotas will then be set based on the project they assign to their jobs, allowing them to work flexibly with multiple labs. However, the report below is from jobs that do not include that information, so users who work within multiple groups have their usage below reported only as a part of their 'home' group/lab.+__NOTE__ Project Slot Quotas will be put into use very soon. Once this is done, each job will be assigned to an SGE project (with the same name as a Project Slot Quota) to track and limit resources by Project Slot Quota. For users who work with multiple groups or labs, this means their quotas will then be set based on the project they assign to their jobs, allowing them to work flexibly with multiple labs. However, the report below is from jobs that do not include that information, so users who work within multiple groups have their usage below reported only as a part of their 'home' group/lab.
  
 **TERMINOLOGY** **TERMINOLOGY**
Line 25: Line 25:
    
 **Slot Equivalent Time** **Slot Equivalent Time**
 +
 The "Slot Equivalent Time" (SET) is the period of time a job runs, multiplied by the SE of the job, reported in units of days or 'divisions'. The "Slot Equivalent Time" (SET) is the period of time a job runs, multiplied by the SE of the job, reported in units of days or 'divisions'.
  
Line 40: Line 41:
  
 ** How to use this report ** ** How to use this report **
-The most important values are the first three columns: '# of non-zero days', and the average and standard deviation values reported under 'Slot Equiv. Time By Day'. 
  
-Check how many days your group had jobs running, and then the SET values. For example if your number of non-zero days is high compared to the number of days in the report, you may want to add the average "SET by Day" to its standard deviation to allow users to generally run the same number of jobs as they have in the past. If your number of non-zero days is low, consider just the "SET by Day" average to save on Slot Group fees.+The most important values are the first three columns: '# of non-zero days', and the average and standard deviation values reported under 'SET By Day'. 
 + 
 +Check how many days your group had jobs running, and then the SET values. For example if your number of non-zero days is high compared to the number of days in the report, you may want to simply add the average "SET by Day" to its standard deviation to allow users to generally run the same number of jobs as they have in the past. If your number of non-zero days is low, consider just the "SET by Day" average to save on Project Slot Quota fees.
  
 You can also consider the "SET By Div" data. If these values are significantly higher, it probably indicates users are running (possibly shorter jobs) during the same time periods, e.g. during the work day. In that case you may want to use these numbers to decide how many slots to request. You can also consider the "SET By Div" data. If these values are significantly higher, it probably indicates users are running (possibly shorter jobs) during the same time periods, e.g. during the work day. In that case you may want to use these numbers to decide how many slots to request.
 +
 +----
  
 **== REPORT == ** **== REPORT == **
 +
 (note, I will refine the output formatting in future reports) (note, I will refine the output formatting in future reports)
  
Line 52: Line 57:
 Period End  :   Fri Nov 11 16:35:37 EST 2016 \\ Period End  :   Fri Nov 11 16:35:37 EST 2016 \\
 Days: 160 \\ Days: 160 \\
-Hours per Division: 4  (for stats reported under "Slot Equiv. Time By Div.") \\+Hours per Division: 4  (for stats reported under "SET By Div.") \\
  
 ^  ^ # of nonzero days (out of 160) ^ SET By Day ^  ^  ^  ^ # of Jobs by day ^  ^  ^  ^  ^ Avg time per job ^ SET By Div ^  ^  ^  ^ Jobs by div. ^  ^  ^  ^ # of nonzero days (out of 160) ^ SET By Day ^  ^  ^  ^ # of Jobs by day ^  ^  ^  ^  ^ Avg time per job ^ SET By Div ^  ^  ^  ^ Jobs by div. ^  ^ 
-^ Group ^  ^ Avg ^ Std ^ Median ^ Max ^ Total_all_days ^ Avg ^ Std ^ Median ^ Max ^ Minutes ^ Avg ^ Std ^ Median ^ Max ^ Avg ^ Std ^ +^ Group ^  ^ Avg ^ Std ^ Median ^ Max ^ Total all days ^ Avg ^ Std ^ Median ^ Max ^ Minutes ^ Avg ^ Std ^ Median ^ Max ^ Avg ^ Std ^ 
 ^ admin | 70.00 | 1.68 | 1.57 | 1.105 | 9.17 | 3756 | 53.66 | 48.30 | 40.000 | 212.00 | 22.522 | 6.78 | 6.48 | 4.354 | 28.34 | 36.15 | 39.91 |  ^ admin | 70.00 | 1.68 | 1.57 | 1.105 | 9.17 | 3756 | 53.66 | 48.30 | 40.000 | 212.00 | 22.522 | 6.78 | 6.48 | 4.354 | 28.34 | 36.15 | 39.91 | 
 +^ Aguirre - TOME | 61.00 | 18.62 | 16.48 | 12.554 | 63.15 | 842382 | 13809.54 | 29972.98 | 32.000 | 148103.00 | .795 | 26.01 | 16.97 | 25.413 | 83.66 | 3219.33 | 6910.41 | 
 +^ Aguirre - MELA | 71.00 | 21.56 | 22.56 | 14.537 | 111.72 | 817578 | 11515.18 | 27772.44 | 32.000 | 135858.00 | 1.206 | 30.72 | 22.39 | 28.032 | 102.18 | 2741.87 | 6467.17 | 
 ^ Ashtari | 15.00 | 7.95 | 5.63 | 5.589 | 16.17 | 514 | 34.27 | 39.40 | 15.000 | 142.00 | 334.067 | 10.37 | 5.54 | 11.000 | 23.28 | 15.52 | 20.57 |  ^ Ashtari | 15.00 | 7.95 | 5.63 | 5.589 | 16.17 | 514 | 34.27 | 39.40 | 15.000 | 142.00 | 334.067 | 10.37 | 5.54 | 11.000 | 23.28 | 15.52 | 20.57 | 
 ^ Avants | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |  ^ Avants | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 
Line 67: Line 74:
 ^ Coslett | 63.00 | 13.82 | 14.76 | 5.699 | 43.41 | 29454 | 467.52 | 1026.17 | 53.000 | 4320.00 | 34.940 | 21.69 | 16.60 | 23.556 | 54.99 | 134.31 | 475.08 |  ^ Coslett | 63.00 | 13.82 | 14.76 | 5.699 | 43.41 | 29454 | 467.52 | 1026.17 | 53.000 | 4320.00 | 34.940 | 21.69 | 16.60 | 23.556 | 54.99 | 134.31 | 475.08 | 
 ^ Davis | 29.00 | 10.61 | 13.81 | 2.833 | 45.53 | 1120 | 38.62 | 65.00 | 10.000 | 250.00 | 136.389 | 15.79 | 18.91 | 5.312 | 91.07 | 13.40 | 28.76 |  ^ Davis | 29.00 | 10.61 | 13.81 | 2.833 | 45.53 | 1120 | 38.62 | 65.00 | 10.000 | 250.00 | 136.389 | 15.79 | 18.91 | 5.312 | 91.07 | 13.40 | 28.76 | 
 +^  ^ # of nonzero days (out of 160) ^ SET By Day ^  ^  ^  ^ # of Jobs by day ^  ^  ^  ^  ^ Avg time per job ^ SET By Div ^  ^  ^  ^ Jobs by div. ^  ^ 
 +^ Group ^  ^ Avg ^ Std ^ Median ^ Max ^ Total all days ^ Avg ^ Std ^ Median ^ Max ^ Minutes ^ Avg ^ Std ^ Median ^ Max ^ Avg ^ Std ^ 
 ^ Detre | 81.00 | 8.49 | 10.67 | 5.945 | 51.79 | 39845 | 491.91 | 1079.01 | 62.000 | 5198.00 | 13.937 | 13.33 | 14.16 | 6.083 | 69.21 | 134.99 | 476.86 |  ^ Detre | 81.00 | 8.49 | 10.67 | 5.945 | 51.79 | 39845 | 491.91 | 1079.01 | 62.000 | 5198.00 | 13.937 | 13.33 | 14.16 | 6.083 | 69.21 | 134.99 | 476.86 | 
 ^ Epstein | 83.00 | 10.75 | 12.66 | 6.832 | 54.24 | 23119 | 278.54 | 576.08 | 95.000 | 3242.00 | 10.614 | 16.13 | 16.78 | 8.416 | 84.40 | 71.80 | 236.20 |  ^ Epstein | 83.00 | 10.75 | 12.66 | 6.832 | 54.24 | 23119 | 278.54 | 576.08 | 95.000 | 3242.00 | 10.614 | 16.13 | 16.78 | 8.416 | 84.40 | 71.80 | 236.20 | 
Line 80: Line 89:
 ^ Kofke | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |  ^ Kofke | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 
 ^ Lerman | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |  ^ Lerman | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 
 +^  ^ # of nonzero days (out of 160) ^ SET By Day ^  ^  ^  ^ # of Jobs by day ^  ^  ^  ^  ^ Avg time per job ^ SET By Div ^  ^  ^  ^ Jobs by div. ^  ^ 
 +^ Group ^  ^ Avg ^ Std ^ Median ^ Max ^ Total all days ^ Avg ^ Std ^ Median ^ Max ^ Minutes ^ Avg ^ Std ^ Median ^ Max ^ Avg ^ Std ^ 
 ^ Loughead | 94.00 | 5.96 | 9.48 | 3.518 | 56.26 | 101696 | 1081.87 | 1848.10 | 125.000 | 8861.00 | 5.441 | 11.23 | 14.39 | 4.705 | 62.45 | 343.90 | 750.02 |  ^ Loughead | 94.00 | 5.96 | 9.48 | 3.518 | 56.26 | 101696 | 1081.87 | 1848.10 | 125.000 | 8861.00 | 5.441 | 11.23 | 14.39 | 4.705 | 62.45 | 343.90 | 750.02 | 
 ^ Mackey | 3.00 | 0.34 | 0.01 | .339 | 0.34 | 3 | 1.00 | 0.00 | 1.000 | 1.00 | 487.716 | 0.76 | 0.44 | 1.000 | 1.00 | 1.00 | 0.00 |  ^ Mackey | 3.00 | 0.34 | 0.01 | .339 | 0.34 | 3 | 1.00 | 0.00 | 1.000 | 1.00 | 487.716 | 0.76 | 0.44 | 1.000 | 1.00 | 1.00 | 0.00 | 
 ^ Medaglia | 40.00 | 4.11 | 5.65 | 2.993 | 29.96 | 338 | 8.45 | 13.98 | 4.000 | 60.00 | 609.949 | 4.73 | 7.47 | 3.000 | 47.62 | 5.41 | 8.64 |  ^ Medaglia | 40.00 | 4.11 | 5.65 | 2.993 | 29.96 | 338 | 8.45 | 13.98 | 4.000 | 60.00 | 609.949 | 4.73 | 7.47 | 3.000 | 47.62 | 5.41 | 8.64 | 
-^ MELA | 71.00 | 21.56 | 22.56 | 14.537 | 111.72 | 817578 | 11515.18 | 27772.44 | 32.000 | 135858.00 | 1.206 | 30.72 | 22.39 | 28.032 | 102.18 | 2741.87 | 6467.17 |  
 ^ Radiology | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |  ^ Radiology | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 
 ^ Rao | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |  ^ Rao | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 
Line 92: Line 102:
 ^ Smith | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 |  ^ Smith | 0.00 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 0 | 0 | 0.00 | 0 | 0 | 
 ^ test1 | 15.00 | 0.07 | 0.09 | .057 | 0.24 | 291 | 19.40 | 32.50 | 6.000 | 121.00 | 2.303 | 0.38 | 0.51 | .048 | 1.47 | 16.17 | 30.34 |  ^ test1 | 15.00 | 0.07 | 0.09 | .057 | 0.24 | 291 | 19.40 | 32.50 | 6.000 | 121.00 | 2.303 | 0.38 | 0.51 | .048 | 1.47 | 16.17 | 30.34 | 
 +^  ^ # of nonzero days (out of 160) ^ SET By Day ^  ^  ^  ^ # of Jobs by day ^  ^  ^  ^  ^ Avg time per job ^ SET By Div ^  ^  ^  ^ Jobs by div. ^  ^ 
 +^ Group ^  ^ Avg ^ Std ^ Median ^ Max ^ Total all days ^ Avg ^ Std ^ Median ^ Max ^ Minutes ^ Avg ^ Std ^ Median ^ Max ^ Avg ^ Std ^ 
 ^ Thompson-Schill | 79.00 | 8.13 | 8.30 | 5.885 | 39.72 | 12873 | 162.95 | 327.71 | 7.000 | 1596.00 | 23.031 | 10.76 | 10.69 | 6.666 | 53.98 | 37.69 | 141.78 |  ^ Thompson-Schill | 79.00 | 8.13 | 8.30 | 5.885 | 39.72 | 12873 | 162.95 | 327.71 | 7.000 | 1596.00 | 23.031 | 10.76 | 10.69 | 6.666 | 53.98 | 37.69 | 141.78 | 
-^ TOME | 61.00 | 18.62 | 16.48 | 12.554 | 63.15 | 842382 | 13809.54 | 29972.98 | 32.000 | 148103.00 | .795 | 26.01 | 16.97 | 25.413 | 83.66 | 3219.33 | 6910.41 |  
 ^ Wehrli | 56.00 | 3.46 | 3.76 | 1.678 | 13.11 | 5921 | 105.73 | 483.69 | 4.000 | 3349.00 | 17.927 | 5.68 | 5.30 | 4.516 | 33.09 | 30.29 | 186.30 |  ^ Wehrli | 56.00 | 3.46 | 3.76 | 1.678 | 13.11 | 5921 | 105.73 | 483.69 | 4.000 | 3349.00 | 17.927 | 5.68 | 5.30 | 4.516 | 33.09 | 30.29 | 186.30 | 
 ^ Wolf | 2.00 | 0 | 0 | 0 | 0.00 | 3 | 1.50 | 0.71 | 2.000 | 2.00 | .260 | 0.00 | 0.00 | .001 | 0.00 | 1.00 | 0.00 |  ^ Wolf | 2.00 | 0 | 0 | 0 | 0.00 | 3 | 1.50 | 0.71 | 2.000 | 2.00 | .260 | 0.00 | 0.00 | .001 | 0.00 | 1.00 | 0.00 | 
 ^ Wolk | 57.00 | 7.69 | 9.97 | 2.509 | 32.37 | 61630 | 1081.23 | 1361.96 | 681.000 | 6086.00 | 6.604 | 15.47 | 16.78 | 7.388 | 74.40 | 361.68 | 503.34 |  ^ Wolk | 57.00 | 7.69 | 9.97 | 2.509 | 32.37 | 61630 | 1081.23 | 1361.96 | 681.000 | 6086.00 | 6.604 | 15.47 | 16.78 | 7.388 | 74.40 | 361.68 | 503.34 | 
 ^ Yushkevich | 114.00 | 17.70 | 17.61 | 12.146 | 64.72 | 170105 | 1492.15 | 2239.33 | 674.000 | 12099.00 | 12.204 | 26.49 | 19.85 | 27.663 | 94.98 | 373.54 | 636.74 |  ^ Yushkevich | 114.00 | 17.70 | 17.61 | 12.146 | 64.72 | 170105 | 1492.15 | 2239.33 | 674.000 | 12099.00 | 12.204 | 26.49 | 19.85 | 27.663 | 94.98 | 373.54 | 636.74 | 
test_table.1481140220.txt.gz · Last modified: 2016/12/07 19:50 by mgstauff

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki