# Python ## Version Update Newer versions of python can focus heavily on performance improvements, some more than others. But be aware regressions for stability or security reasons may always be the case. As a general rule, Python 3.9 and Python 3.11 both had a strong focus on performance improvements. Python 3.12 is looking promising but has yet to have widespread testing, adoption and deployment. A simple comparison is available at [speed.python.org](https://speed.python.org/comparison/?exe=12%2BL%2B3.11%2C12%2BL%2B3.12%2C12%2BL%2B3.10%2C12%2BL%2B3.9%2C12%2BL%2B3.8&ben=746&env=1&hor=false&bas=none&chart=normal+bars). [Djangobench](https://github.com/django/djangobench/tree/master) is the source of synthetic benchmarks and a useful tool for running comparisons. Below are some examples to inform your investigations. Keep in mind while a 1.2x faster result is significant, it's only one step of the process, Celery, SQL, Redis, and many other factors will influence the endresult, and this _python_ speed improvement will not translate 1:1 into real world performance. ### Django 4.0.10 - Djangobench 0.10.0 - Django 4.0.10 - Python 3.8.18 vs Python 3.11.5 ```bash joel@METABOX:~/djangobench/django$ djangobench --vcs=none --control=. --experiment=. --control-python=/home/joel/djangobench/py38/bin/python --experiment-python=/home/joel/djangobench/py311/bin/python -r /home/joel/djangobench/results -t 500 Running all benchmarks Recording data to '/home/joel/djangobench/results' Control: Django 4.0.10 (in .) Experiment: Django 4.0.10 (in .) Running 'multi_value_dict' benchmark ... Min: 0.000014 -> 0.000013: 1.0304x faster Avg: 0.000183 -> 0.000133: 1.3698x faster Significant (t=9.325958) Stddev: 0.00010 -> 0.00007: 1.3791x smaller (N = 500) Running 'query_values' benchmark ... Min: 0.000079 -> 0.000070: 1.1308x faster Avg: 0.000084 -> 0.000074: 1.1267x faster Significant (t=19.174361) Stddev: 0.00001 -> 0.00001: 1.0255x larger (N = 500) Running 'query_delete' benchmark ... Min: 0.000082 -> 0.000074: 1.1145x faster Avg: 0.000086 -> 0.000078: 1.0987x faster Significant (t=17.504085) Stddev: 0.00001 -> 0.00001: 1.1888x smaller (N = 500) Running 'query_select_related' benchmark ... Min: 0.016771 -> 0.013520: 1.2405x faster Avg: 0.017897 -> 0.014149: 1.2649x faster Significant (t=40.942990) Stddev: 0.00190 -> 0.00077: 2.4535x smaller (N = 500) Running 'query_aggregate' benchmark ... Min: 0.000092 -> 0.000083: 1.1105x faster Avg: 0.000100 -> 0.000090: 1.1107x faster Significant (t=9.967204) Stddev: 0.00002 -> 0.00001: 1.5003x smaller (N = 500) Running 'query_raw_deferred' benchmark ... Min: 0.004157 -> 0.003563: 1.1666x faster Avg: 0.004626 -> 0.003809: 1.2143x faster Significant (t=12.325104) Stddev: 0.00121 -> 0.00086: 1.4047x smaller (N = 500) Running 'query_get_or_create' benchmark ... Min: 0.000412 -> 0.000362: 1.1385x faster Avg: 0.000458 -> 0.000407: 1.1259x faster Significant (t=14.169322) Stddev: 0.00006 -> 0.00005: 1.1306x smaller (N = 500) Running 'query_values_list' benchmark ... Min: 0.000080 -> 0.000071: 1.1231x faster Avg: 0.000089 -> 0.000076: 1.1706x faster Significant (t=18.298942) Stddev: 0.00001 -> 0.00001: 1.9398x smaller (N = 500) Running 'url_resolve_flat_i18n_off' benchmark ... Min: 0.055764 -> 0.045370: 1.2291x faster Avg: 0.057670 -> 0.047020: 1.2265x faster Significant (t=111.187780) Stddev: 0.00206 -> 0.00059: 3.4618x smaller (N = 500) Running 'qs_filter_chaining' benchmark ... Min: 0.000236 -> 0.000196: 1.2034x faster Avg: 0.000248 -> 0.000206: 1.2041x faster Significant (t=44.893544) Stddev: 0.00002 -> 0.00001: 1.0833x smaller (N = 500) Running 'template_render' benchmark ... Min: 0.000933 -> 0.000712: 1.3110x faster Avg: 0.001003 -> 0.000777: 1.2909x faster Significant (t=8.379095) Stddev: 0.00043 -> 0.00042: 1.0287x smaller (N = 500) Running 'query_get' benchmark ... Min: 0.000259 -> 0.000230: 1.1259x faster Avg: 0.000282 -> 0.000238: 1.1829x faster Significant (t=42.267305) Stddev: 0.00002 -> 0.00001: 1.7842x smaller (N = 500) Running 'query_none' benchmark ... Min: 0.000053 -> 0.000045: 1.1830x faster Avg: 0.000056 -> 0.000049: 1.1449x faster Significant (t=16.426843) Stddev: 0.00001 -> 0.00001: 1.1267x larger (N = 500) Running 'query_complex_filter' benchmark ... Min: 0.000039 -> 0.000034: 1.1527x faster Avg: 0.000041 -> 0.000037: 1.1091x faster Significant (t=13.582718) Stddev: 0.00000 -> 0.00001: 1.5373x larger (N = 500) Running 'query_filter' benchmark ... Min: 0.000127 -> 0.000112: 1.1288x faster Avg: 0.000133 -> 0.000119: 1.1228x faster Significant (t=22.727829) Stddev: 0.00001 -> 0.00001: 1.1771x smaller (N = 500) Running 'template_render_simple' benchmark ... Min: 0.000030 -> 0.000024: 1.2405x faster Avg: 0.000035 -> 0.000029: 1.2042x faster Not significant Stddev: 0.00007 -> 0.00005: 1.3190x smaller (N = 500) Running 'default_middleware' benchmark ... Min: -0.000047 -> -0.000054: 0.8624x faster Avg: 0.000017 -> 0.000017: 1.0032x faster Not significant Stddev: 0.00037 -> 0.00037: 1.0091x larger (N = 500) Running 'query_annotate' benchmark ... Min: 0.000186 -> 0.000162: 1.1505x faster Avg: 0.000207 -> 0.000178: 1.1660x faster Significant (t=16.516089) Stddev: 0.00003 -> 0.00003: 1.1403x smaller (N = 500) Running 'raw_sql' benchmark ... Min: 0.000015 -> 0.000013: 1.1070x faster Avg: 0.000017 -> 0.000014: 1.1676x faster Significant (t=13.598519) Stddev: 0.00000 -> 0.00000: 2.3503x smaller (N = 500) Running 'url_resolve_flat' benchmark ... Min: 0.056378 -> 0.044772: 1.2592x faster Avg: 0.058268 -> 0.046656: 1.2489x faster Significant (t=197.176590) Stddev: 0.00121 -> 0.00051: 2.3665x smaller (N = 500) Running 'l10n_render' benchmark ... Min: 0.001097 -> 0.000727: 1.5092x faster Avg: 0.001160 -> 0.000768: 1.5101x faster Significant (t=36.971179) Stddev: 0.00019 -> 0.00014: 1.2946x smaller (N = 500) Running 'query_count' benchmark ... Min: 0.000083 -> 0.000073: 1.1302x faster Avg: 0.000091 -> 0.000079: 1.1640x faster Significant (t=15.049336) Stddev: 0.00002 -> 0.00001: 1.6661x smaller (N = 500) Running 'model_delete' benchmark ... Min: 0.000123 -> 0.000105: 1.1701x faster Avg: 0.000135 -> 0.000119: 1.1396x faster Significant (t=17.781816) Stddev: 0.00001 -> 0.00002: 1.1990x larger (N = 500) Running 'query_iterator' benchmark ... Min: 0.000102 -> 0.000088: 1.1605x faster Avg: 0.000108 -> 0.000093: 1.1598x faster Significant (t=23.872009) Stddev: 0.00001 -> 0.00001: 1.1366x smaller (N = 500) Running 'template_compilation' benchmark ... Min: 0.000155 -> 0.000129: 1.2015x faster Avg: 0.000169 -> 0.000137: 1.2317x faster Significant (t=6.119618) Stddev: 0.00009 -> 0.00007: 1.4162x smaller (N = 500) Running 'query_all_multifield' benchmark ... Min: 0.014582 -> 0.012509: 1.1658x faster Avg: 0.015715 -> 0.013337: 1.1783x faster Significant (t=19.183517) Stddev: 0.00207 -> 0.00184: 1.1241x smaller (N = 500) Running 'query_prefetch_related' benchmark ... Min: 0.014293 -> 0.012157: 1.1758x faster Avg: 0.015467 -> 0.013276: 1.1650x faster Significant (t=20.607411) Stddev: 0.00176 -> 0.00160: 1.0952x smaller (N = 500) Running 'query_all_converters' benchmark ... Min: 0.000536 -> 0.000464: 1.1554x faster Avg: 0.000563 -> 0.000486: 1.1595x faster Significant (t=38.503433) Stddev: 0.00004 -> 0.00002: 1.6468x smaller (N = 500) Running 'query_distinct' benchmark ... Min: 0.000106 -> 0.000092: 1.1583x faster Avg: 0.000127 -> 0.000096: 1.3223x faster Significant (t=27.798102) Stddev: 0.00002 -> 0.00001: 3.7187x smaller (N = 500) Running 'query_dates' benchmark ... Min: 0.000249 -> 0.000209: 1.1953x faster Avg: 0.000275 -> 0.000228: 1.2056x faster Significant (t=30.785168) Stddev: 0.00003 -> 0.00002: 1.0854x smaller (N = 500) Running 'model_save_existing' benchmark ... Min: 0.003526 -> 0.003094: 1.1397x faster Avg: 0.003723 -> 0.003212: 1.1591x faster Significant (t=47.274918) Stddev: 0.00018 -> 0.00016: 1.1817x smaller (N = 500) Running 'query_delete_related' benchmark ... Min: 0.000120 -> 0.000103: 1.1655x faster Avg: 0.000132 -> 0.000111: 1.1815x faster Significant (t=6.428771) Stddev: 0.00005 -> 0.00004: 1.2149x smaller (N = 500) Running 'url_reverse' benchmark ... Min: 0.000062 -> 0.000060: 1.0318x faster Avg: 0.000072 -> 0.000068: 1.0622x faster Not significant Stddev: 0.00006 -> 0.00005: 1.0531x smaller (N = 500) Running 'query_latest' benchmark ... Min: 0.000136 -> 0.000118: 1.1454x faster Avg: 0.000155 -> 0.000129: 1.2008x faster Significant (t=8.372115) Stddev: 0.00007 -> 0.00001: 5.1365x smaller (N = 500) Running 'form_create' benchmark ... Min: 0.000015 -> 0.000013: 1.2319x faster Avg: 0.000019 -> 0.000015: 1.2739x faster Significant (t=4.158080) Stddev: 0.00002 -> 0.00001: 1.1449x smaller (N = 500) Running 'query_update' benchmark ... Min: 0.000047 -> 0.000041: 1.1323x faster Avg: 0.000052 -> 0.000044: 1.1721x faster Significant (t=18.470635) Stddev: 0.00001 -> 0.00000: 1.6104x smaller (N = 500) Running 'query_in_bulk' benchmark ... Min: 0.000152 -> 0.000136: 1.1193x faster Avg: 0.000173 -> 0.000147: 1.1735x faster Significant (t=16.901845) Stddev: 0.00003 -> 0.00001: 2.1199x smaller (N = 500) Running 'url_resolve_nested' benchmark ... Min: 0.000043 -> 0.000034: 1.2871x faster Avg: 0.000075 -> 0.000047: 1.6049x faster Not significant Stddev: 0.00066 -> 0.00023: 2.8387x smaller (N = 500) Running 'model_creation' benchmark ... Min: 0.000077 -> 0.000066: 1.1579x faster Avg: 0.000088 -> 0.000072: 1.2205x faster Significant (t=10.514202) Stddev: 0.00003 -> 0.00001: 3.1410x smaller (N = 500) Running 'query_order_by' benchmark ... Min: 0.000135 -> 0.000124: 1.0945x faster Avg: 0.000145 -> 0.000133: 1.0902x faster Significant (t=13.574502) Stddev: 0.00001 -> 0.00001: 1.1586x smaller (N = 500) Running 'startup' benchmark ... Skipped: Django 1.9 and later has changed app loading. This benchmark needs fixing anyway. Running 'form_clean' benchmark ... Min: 0.000005 -> 0.000003: 1.4696x faster Avg: 0.000006 -> 0.000004: 1.4931x faster Significant (t=11.263253) Stddev: 0.00000 -> 0.00000: 2.2571x smaller (N = 500) Running 'locale_from_request' benchmark ... Min: 0.000076 -> 0.000082: 1.0895x slower Avg: 0.000083 -> 0.000090: 1.0877x slower Not significant Stddev: 0.00009 -> 0.00006: 1.6230x smaller (N = 500) Running 'query_exists' benchmark ... Min: 0.000243 -> 0.000214: 1.1399x faster Avg: 0.000262 -> 0.000227: 1.1571x faster Significant (t=27.797738) Stddev: 0.00002 -> 0.00002: 1.2601x smaller (N = 500) Running 'query_values_10000' benchmark ... Min: 0.005755 -> 0.005269: 1.0923x faster Avg: 0.006184 -> 0.005587: 1.1067x faster Significant (t=10.895954) Stddev: 0.00094 -> 0.00079: 1.1902x smaller (N = 500) Running 'query_exclude' benchmark ... Min: 0.000159 -> 0.000141: 1.1256x faster Avg: 0.000177 -> 0.000151: 1.1741x faster Significant (t=23.556200) Stddev: 0.00002 -> 0.00001: 1.8250x smaller (N = 500) Running 'query_raw' benchmark ... Min: 0.005619 -> 0.004860: 1.1562x faster Avg: 0.006181 -> 0.005041: 1.2263x faster Significant (t=18.008590) Stddev: 0.00121 -> 0.00074: 1.6376x smaller (N = 500) Running 'url_resolve' benchmark ... Min: 0.004666 -> 0.004233: 1.1023x faster Avg: 0.004920 -> 0.004347: 1.1318x faster Significant (t=24.865249) Stddev: 0.00049 -> 0.00016: 3.1507x smaller (N = 500) Running 'model_save_new' benchmark ... Min: 0.003420 -> 0.003105: 1.1014x faster Avg: 0.003610 -> 0.003217: 1.1221x faster Significant (t=42.956103) Stddev: 0.00017 -> 0.00011: 1.6304x smaller (N = 500) Running 'query_all' benchmark ... Min: 0.008101 -> 0.007077: 1.1447x faster Avg: 0.009006 -> 0.007936: 1.1348x faster Significant (t=9.981534) Stddev: 0.00171 -> 0.00168: 1.0215x smaller (N = 500) ``` ### Django 4.2.6 - Djangobench 0.10.0 - Django 4.0.10 - Python 3.8.18 vs Python 3.11.5 ```bash joel@METABOX:~/djangobench/django$ djangobench --vcs=none --control=. --experiment=. --control-python=/home/joel/djangobench/py38/bin/python --experiment-python=/home/joel/djangobench/py311/bin/python -r /home/joel/djangobench/results -t 500 Running all benchmarks Recording data to '/home/joel/djangobench/results' Control: Django 4.2.6 (in .) Experiment: Django 4.2.6 (in .) Running 'multi_value_dict' benchmark ... Min: -0.000004 -> 0.000013: -3.0336x slower Avg: 0.000182 -> 0.000133: 1.3680x faster Significant (t=9.151616) Stddev: 0.00010 -> 0.00007: 1.3826x smaller (N = 500) Running 'query_values' benchmark ... Min: 0.000082 -> 0.000072: 1.1485x faster Avg: 0.000086 -> 0.000075: 1.1462x faster Significant (t=30.114973) Stddev: 0.00001 -> 0.00001: 1.0258x larger (N = 500) Running 'query_delete' benchmark ... Min: 0.000080 -> 0.000071: 1.1169x faster Avg: 0.000086 -> 0.000077: 1.1088x faster Significant (t=13.459411) Stddev: 0.00001 -> 0.00001: 1.0008x smaller (N = 500) Running 'query_select_related' benchmark ... Min: 0.016889 -> 0.013513: 1.2498x faster Avg: 0.018370 -> 0.013885: 1.3230x faster Significant (t=48.921967) Stddev: 0.00196 -> 0.00061: 3.2174x smaller (N = 500) Running 'query_aggregate' benchmark ... Min: 0.000167 -> 0.000153: 1.0904x faster Avg: 0.000182 -> 0.000165: 1.1029x faster Significant (t=12.685517) Stddev: 0.00002 -> 0.00002: 1.3019x smaller (N = 500) Running 'query_raw_deferred' benchmark ... Min: 0.004160 -> 0.003674: 1.1323x faster Avg: 0.004596 -> 0.003888: 1.1820x faster Significant (t=11.504156) Stddev: 0.00117 -> 0.00073: 1.5957x smaller (N = 500) Running 'query_get_or_create' benchmark ... Min: 0.000421 -> 0.000356: 1.1823x faster Avg: 0.000470 -> 0.000392: 1.2011x faster Significant (t=14.613017) Stddev: 0.00008 -> 0.00009: 1.0954x larger (N = 500) Running 'query_values_list' benchmark ... Min: 0.000080 -> 0.000070: 1.1395x faster Avg: 0.000085 -> 0.000075: 1.1202x faster Significant (t=20.300988) Stddev: 0.00001 -> 0.00001: 1.0537x smaller (N = 500) Running 'url_resolve_flat_i18n_off' benchmark ... Min: 0.056031 -> 0.045854: 1.2219x faster Avg: 0.057048 -> 0.048370: 1.1794x faster Significant (t=106.668460) Stddev: 0.00117 -> 0.00139: 1.1819x larger (N = 500) Running 'qs_filter_chaining' benchmark ... Min: 0.000247 -> 0.000205: 1.2080x faster Avg: 0.000267 -> 0.000219: 1.2211x faster Significant (t=38.507950) Stddev: 0.00002 -> 0.00002: 1.0252x larger (N = 500) Running 'template_render' benchmark ... Min: 0.000956 -> 0.000761: 1.2550x faster Avg: 0.001061 -> 0.000862: 1.2302x faster Significant (t=6.128572) Stddev: 0.00052 -> 0.00051: 1.0109x smaller (N = 500) Running 'query_get' benchmark ... Min: 0.000268 -> 0.000235: 1.1388x faster Avg: 0.000293 -> 0.000256: 1.1411x faster Significant (t=24.002331) Stddev: 0.00002 -> 0.00003: 1.2917x larger (N = 500) Running 'query_none' benchmark ... Min: 0.000055 -> 0.000050: 1.1079x faster Avg: 0.000061 -> 0.000055: 1.1183x faster Significant (t=3.149707) Stddev: 0.00003 -> 0.00004: 1.3162x larger (N = 500) Running 'query_complex_filter' benchmark ... Min: 0.000040 -> 0.000034: 1.1777x faster Avg: 0.000042 -> 0.000038: 1.1267x faster Significant (t=15.246074) Stddev: 0.00000 -> 0.00001: 1.5250x larger (N = 500) Running 'query_filter' benchmark ... Min: 0.000131 -> 0.000116: 1.1288x faster Avg: 0.000139 -> 0.000127: 1.0907x faster Significant (t=14.448319) Stddev: 0.00001 -> 0.00001: 1.2281x larger (N = 500) Running 'template_render_simple' benchmark ... Min: 0.000031 -> 0.000024: 1.2650x faster Avg: 0.000037 -> 0.000029: 1.2895x faster Significant (t=2.094800) Stddev: 0.00007 -> 0.00005: 1.3630x smaller (N = 500) Running 'default_middleware' benchmark ... Min: -0.000037 -> -0.000060: 0.6180x faster Avg: 0.000001 -> 0.000001: 1.0056x slower Not significant Stddev: 0.00002 -> 0.00002: 1.0915x smaller (N = 500) Running 'query_annotate' benchmark ... Min: 0.000192 -> 0.000173: 1.1122x faster Avg: 0.000206 -> 0.000185: 1.1134x faster Significant (t=17.849733) Stddev: 0.00002 -> 0.00002: 1.0456x smaller (N = 500) Running 'raw_sql' benchmark ... Min: 0.000013 -> 0.000012: 1.0839x faster Avg: 0.000015 -> 0.000014: 1.0882x faster Significant (t=4.252084) Stddev: 0.00001 -> 0.00000: 1.5868x smaller (N = 500) Running 'url_resolve_flat' benchmark ... Min: 0.055540 -> 0.046018: 1.2069x faster Avg: 0.058030 -> 0.048408: 1.1988x faster Significant (t=98.852976) Stddev: 0.00157 -> 0.00151: 1.0444x smaller (N = 500) Running 'l10n_render' benchmark ... Min: 0.001604 -> 0.001253: 1.2797x faster Avg: 0.001684 -> 0.001304: 1.2918x faster Significant (t=37.535402) Stddev: 0.00017 -> 0.00015: 1.1476x smaller (N = 500) Running 'query_count' benchmark ... Min: 0.000176 -> 0.000165: 1.0631x faster Avg: 0.000189 -> 0.000176: 1.0755x faster Significant (t=12.229046) Stddev: 0.00002 -> 0.00002: 1.0395x larger (N = 500) Running 'model_delete' benchmark ... Min: 0.000122 -> 0.000104: 1.1743x faster Avg: 0.000152 -> 0.000115: 1.3227x faster Significant (t=19.812953) Stddev: 0.00004 -> 0.00001: 2.6554x smaller (N = 500) Running 'query_iterator' benchmark ... Min: 0.000108 -> 0.000094: 1.1518x faster Avg: 0.000119 -> 0.000098: 1.2203x faster Significant (t=21.984884) Stddev: 0.00002 -> 0.00001: 2.7750x smaller (N = 500) Running 'template_compilation' benchmark ... Min: 0.000164 -> 0.000148: 1.1034x faster Avg: 0.000184 -> 0.000162: 1.1386x faster Significant (t=4.665298) Stddev: 0.00008 -> 0.00007: 1.2952x smaller (N = 500) Running 'query_all_multifield' benchmark ... Min: 0.014802 -> 0.012188: 1.2144x faster Avg: 0.016029 -> 0.013294: 1.2057x faster Significant (t=21.516971) Stddev: 0.00210 -> 0.00191: 1.0984x smaller (N = 500) Running 'query_prefetch_related' benchmark ... Min: 0.013401 -> 0.011583: 1.1569x faster Avg: 0.014822 -> 0.013366: 1.1090x faster Significant (t=11.422100) Stddev: 0.00170 -> 0.00229: 1.3410x larger (N = 500) Running 'query_all_converters' benchmark ... Min: 0.000499 -> 0.000429: 1.1618x faster Avg: 0.000531 -> 0.000455: 1.1670x faster Significant (t=42.716720) Stddev: 0.00002 -> 0.00003: 1.5394x larger (N = 500) Running 'query_distinct' benchmark ... Min: 0.000108 -> 0.000095: 1.1397x faster Avg: 0.000116 -> 0.000100: 1.1646x faster Significant (t=25.915629) Stddev: 0.00001 -> 0.00001: 1.1204x larger (N = 500) Running 'query_dates' benchmark ... Min: 0.000333 -> 0.000290: 1.1490x faster Avg: 0.000365 -> 0.000326: 1.1207x faster Significant (t=18.213858) Stddev: 0.00003 -> 0.00003: 1.0118x larger (N = 500) Running 'model_save_existing' benchmark ... Min: 0.003455 -> 0.003081: 1.1215x faster Avg: 0.003764 -> 0.003326: 1.1316x faster Significant (t=32.229651) Stddev: 0.00023 -> 0.00020: 1.1398x smaller (N = 500) Running 'query_delete_related' benchmark ... Min: 0.000122 -> 0.000102: 1.1946x faster Avg: 0.000131 -> 0.000113: 1.1564x faster Significant (t=5.027485) Stddev: 0.00005 -> 0.00006: 1.4129x larger (N = 500) Running 'url_reverse' benchmark ... Min: 0.000068 -> 0.000067: 1.0193x faster Avg: 0.000075 -> 0.000074: 1.0157x faster Not significant Stddev: 0.00006 -> 0.00005: 1.1543x smaller (N = 500) Running 'query_latest' benchmark ... Min: 0.000147 -> 0.000138: 1.0631x faster Avg: 0.000167 -> 0.000148: 1.1277x faster Significant (t=11.353029) Stddev: 0.00003 -> 0.00002: 1.6091x smaller (N = 500) Running 'form_create' benchmark ... Min: 0.000016 -> 0.000013: 1.2659x faster Avg: 0.000020 -> 0.000015: 1.2770x faster Significant (t=3.482649) Stddev: 0.00002 -> 0.00002: 1.0947x larger (N = 500) Running 'query_update' benchmark ... Min: 0.000047 -> 0.000043: 1.0971x faster Avg: 0.000050 -> 0.000046: 1.0691x faster Significant (t=9.363513) Stddev: 0.00001 -> 0.00000: 1.2636x smaller (N = 500) Running 'query_in_bulk' benchmark ... Min: 0.000157 -> 0.000143: 1.0970x faster Avg: 0.000178 -> 0.000162: 1.0981x faster Significant (t=9.031182) Stddev: 0.00002 -> 0.00003: 1.5173x larger (N = 500) Running 'url_resolve_nested' benchmark ... Min: 0.000046 -> 0.000038: 1.2179x faster Avg: 0.000075 -> 0.000052: 1.4505x faster Not significant Stddev: 0.00059 -> 0.00024: 2.4300x smaller (N = 500) Running 'model_creation' benchmark ... Min: 0.000071 -> 0.000065: 1.1058x faster Avg: 0.000079 -> 0.000073: 1.0876x faster Significant (t=2.786580) Stddev: 0.00003 -> 0.00004: 1.1518x larger (N = 500) Running 'query_order_by' benchmark ... Min: 0.000146 -> 0.000128: 1.1407x faster Avg: 0.000154 -> 0.000138: 1.1206x faster Significant (t=14.021341) Stddev: 0.00002 -> 0.00002: 1.2540x larger (N = 500) Running 'startup' benchmark ... Skipped: Django 1.9 and later has changed app loading. This benchmark needs fixing anyway. Running 'form_clean' benchmark ... Min: 0.000005 -> 0.000004: 1.4613x faster Avg: 0.000006 -> 0.000004: 1.3654x faster Significant (t=12.763128) Stddev: 0.00000 -> 0.00000: 1.1666x larger (N = 500) Running 'locale_from_request' benchmark ... Min: 0.000097 -> 0.000108: 1.1090x slower Avg: 0.000108 -> 0.000120: 1.1178x slower Significant (t=-3.057677) Stddev: 0.00007 -> 0.00006: 1.1186x smaller (N = 500) Running 'query_exists' benchmark ... Min: 0.000273 -> 0.000234: 1.1698x faster Avg: 0.000290 -> 0.000248: 1.1686x faster Significant (t=39.518859) Stddev: 0.00002 -> 0.00002: 1.2025x smaller (N = 500) Running 'query_values_10000' benchmark ... Min: 0.005601 -> 0.005298: 1.0571x faster Avg: 0.006023 -> 0.005691: 1.0583x faster Significant (t=6.167352) Stddev: 0.00082 -> 0.00088: 1.0752x larger (N = 500) Running 'query_exclude' benchmark ... Min: 0.000159 -> 0.000140: 1.1367x faster Avg: 0.000165 -> 0.000149: 1.1020x faster Significant (t=19.643154) Stddev: 0.00001 -> 0.00001: 1.2636x larger (N = 500) Running 'query_raw' benchmark ... Min: 0.005764 -> 0.004630: 1.2450x faster Avg: 0.006169 -> 0.004881: 1.2638x faster Significant (t=20.996453) Stddev: 0.00109 -> 0.00083: 1.3105x smaller (N = 500) Running 'url_resolve' benchmark ... Min: 0.004928 -> 0.004597: 1.0721x faster Avg: 0.005217 -> 0.004716: 1.1063x faster Significant (t=46.893945) Stddev: 0.00022 -> 0.00010: 2.2192x smaller (N = 500) Running 'model_save_new' benchmark ... Min: 0.003404 -> 0.003012: 1.1301x faster Avg: 0.003494 -> 0.003105: 1.1251x faster Significant (t=45.888484) Stddev: 0.00014 -> 0.00013: 1.0298x smaller (N = 500) Running 'query_all' benchmark ... Min: 0.007971 -> 0.007085: 1.1250x faster Avg: 0.009091 -> 0.008147: 1.1159x faster Significant (t=7.583074) Stddev: 0.00183 -> 0.00210: 1.1518x larger (N = 500) ```