From 1cdb6cb9079f5c0e2b709ab90294b08218b50831 Mon Sep 17 00:00:00 2001 From: Wouter de Jong Date: Sat, 17 Oct 2020 15:19:01 +0200 Subject: [PATCH] Added limiter diagrams --- _images/rate_limiter/fixed_window.svg | 84 ++++++++++++++++++ _images/rate_limiter/sliding_window.svg | 65 ++++++++++++++ _images/rate_limiter/token_bucket.svg | 83 +++++++++++++++++ _images/sources/rate_limiter/fixed_window.dia | Bin 0 -> 2356 bytes .../sources/rate_limiter/sliding_window.dia | Bin 0 -> 2190 bytes _images/sources/rate_limiter/token_bucket.dia | Bin 0 -> 2752 bytes rate_limiter.rst | 52 +++++++++-- 7 files changed, 275 insertions(+), 9 deletions(-) create mode 100644 _images/rate_limiter/fixed_window.svg create mode 100644 _images/rate_limiter/sliding_window.svg create mode 100644 _images/rate_limiter/token_bucket.svg create mode 100644 _images/sources/rate_limiter/fixed_window.dia create mode 100644 _images/sources/rate_limiter/sliding_window.dia create mode 100644 _images/sources/rate_limiter/token_bucket.dia diff --git a/_images/rate_limiter/fixed_window.svg b/_images/rate_limiter/fixed_window.svg new file mode 100644 index 00000000000..83d5f6e79ac --- /dev/null +++ b/_images/rate_limiter/fixed_window.svg @@ -0,0 +1,84 @@ + + + + + + + + + + + + + + + + + + + + + + 10:00 + + + 10:30 + + + 11:00 + + + 11:30 + + + 12:00 + + + + + + + + 12:30 + + + 13:00 + + + + + + + + + + + + + + + + + + + + + + 1 hour window + + + 1 hour window + + + + + + 1 hour window + + + + + 13:15 + + + diff --git a/_images/rate_limiter/sliding_window.svg b/_images/rate_limiter/sliding_window.svg new file mode 100644 index 00000000000..2c565615441 --- /dev/null +++ b/_images/rate_limiter/sliding_window.svg @@ -0,0 +1,65 @@ + + + + + + + + + + 10:00 + + + 10:30 + + + 11:00 + + + 11:30 + + + 12:00 + + + + + + 12:30 + + + 13:00 + + + + + + + + + + + + + + + + + + + + + + 1 hour window + + + + + + 13:15 + + + + + + diff --git a/_images/rate_limiter/token_bucket.svg b/_images/rate_limiter/token_bucket.svg new file mode 100644 index 00000000000..29d6fc8f103 --- /dev/null +++ b/_images/rate_limiter/token_bucket.svg @@ -0,0 +1,83 @@ + + + + 10:00 + + + 10:30 + + + 11:00 + + + 11:30 + + + 12:00 + + + + + + + + 12:30 + + + 13:00 + + + + + + + + + + + + + + + + + + + + + + + + + + + 13:15 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/_images/sources/rate_limiter/fixed_window.dia b/_images/sources/rate_limiter/fixed_window.dia new file mode 100644 index 0000000000000000000000000000000000000000..16282a2dcce2dc3cf58a40be13992366c2a10f48 GIT binary patch literal 2356 zcmV-43Cs2$iwFP!000021MQt#liD~I$KUfQT;#ReSQp>Y=}ygT)$BfO?WQW3=PrTO z;2C2xwrRpV>}OxuTpElG23hKuI#eYAB6KYI==VRCb>z=qe#|1{m8V&l#E%1HS_6Z} z<0J^<$>ZSn@4wx9gD+1XJ_jNDB>qfNHZ#O8;;i`eIGE=7{L^r_TrSOMm9acY%_v-$ z86Wic8q{$)<2J2Fra^oaQ zQsb3HkAsgdo7Z5tsc3jm(}~*WY{Ew=XMZ+VZi$!gUsw5@r`3vQ^CSyJk$g2jD$-C- z@%LVtO{q*2iYHGWf1w}OhqU?Pt**KQZ8Vb4SUL&gV>_aEqIFkb)592x?y!B&w_NI? zZJ%6jZlu>-q}N=k*IbxA&yzGyS(qQ293@G_S-iIT!lH~{ZzvnHNch;Pc1bkjCCu}r zqXw+M-VL`)hwdZ|gVQ7TphQD0!Akraj@6!%Wfi5qx!R{ZoJ{i*)n8ZlC`kjJo;!BSMArI+sBadA zb(xU@0M9j-f9J>jC00Z{%b&A+74fP89xrC=+Ua#w&sh*0M$1R%<4RY``Hy^XCN8Oz zY3k(iqVb?y<@&PW>hc86o5K4f&g%>n-y1I?P>xpOcvzebjPiVxtXu!|Up#u{c{pbK zY`^>njs{=tIq|H+7nL)5NZ~qcQEU>|zN03k)NB_=J6JP38uAfa+iYNencpCF$gpCF$d$tQ74 z5KrymiDktTJF2A<+or#fPHbArrBkXwAe0=WGHr)uDTxmABI@shKu*vHen=r&p8rkFmn|i^ffAVC;#jnN!PG5QY zQvA=izZNXzHJI|`?y~zDtYFUKr*J|+xiq(JEv`ZN}WH#`jlR-okP+IQIu$a6U7H` zqWyvsfyHwg;Y0vVx3_o>$tOp78s<*D-Q1~v3v;I)%$>reGh8~urL*cw=YNZHR=0bl zblMp%(|jCL^-dzf1hpTiYm&m)mDY-#w4Zq#9Bl*MqE$74kSo=6F;>R&>M_(kOyK>& z*^#rk-RxWAsNL=j-xXWFJ&1S31(BuP)v9f3D&m5hwR5vB@<7~TU}MFJAZ}r!*cr(a zZ#>n0R@12FQZ7KPrMLOrYn_7oc{5n>iia9o7IfZwoa~57fL?Q1bw4mwB7p&4NyqP%A8~p(fsGf!a}< znu-9_TDqC^jRJ}^K&>!RLQTBU0kyhDHJ1XYwe>c?n}wiCp+n{oh>tB001yD}0?^T37)1F_7-jVD+(7g%)nNdCi2eco0RL{_PrAwGB+7F+ zOZfY@0DlkQ5AcV`AK?Ekn;;SS2QT|~*44N3+NL}a&mnjBmWe%zy>jWcgNS=4Xw#Fe zMvSQ#_r@}egJcO?j_7&@bJaCrzmZ;UhD797D>|mF8dHRrUIkOWi4gGsQ-GQ$^@4JXM6iQ$^r%8s<{ra{4EjOLe_7^0`{;|}Zrn`uBa`fdM<%&_dt?%_A>h&#a5?bEq@OP0T7Qn% ail)*r;5Ny=tVAN~(zH@RP^pI)w3?7e? zAdJTky+6NwzN5X5k2fEJki8dw#wnW`;u~>Re0u0j@_hEb-(M^iX0*&$o}^|J&drSX z|7B6c`l3+3_jqF%yA=d1XT`PkRhH*zIGpF)h}o1s^oDHoeViuqIOwfPt;>y)C`pZH z7CrRdJ*{88{<@<6UQMsmK4W7(OgZ~rU%4e-zJFQeGoBt+Je?(3D2n9E*-??2dWyey z%B)LeqEI}3eD@Q1x4NbEg_pXjZnRcNK4s}RjF07r-icOSVVl&pZP6W$PkqZH9@_Lt zb8$nx;v&7`QoZ8BY%ojGJY`{iEOMA65ohsA>Jy7Hez~A*#3G?%ueD2}8BbxJCoMH# zb$L77nhxD@8V0YA+`SStwFFD?Z#Yu>Ocr5~PX<3;dnT4?JF)_n>I6K8SvZV%RU5)M zmsYDy^~-juKU9R$R=j*&Qt?)`)f%IKa)&#Y7^Cq#40v`zr}xX$)HGQa>!0fP!*Y^c z6NK3;V$0W?a9F6OE*9sZKpD$;pqQ93o89Hf-2pDDB<1P61wqO1FFwlEikc(~4Gk)! zjFLDO%@N+{-7FEKHPdV!R+kPu)l#fbGM=t)_0x-wYOHCqZoRQ?0Nc65M0=h$pLX

^@POa0-qFbJ>flzQUI2+@d{l z*Y#GNfZ_14{ZA_gj zQT2p}<4N9n-NPgeczWu_EfZO*7b3oekc;fg#Dzkm)4{%>s+yDq!P9W~Vyc=6DMtUG zV(xdEGfkujXO<7Ld>QdW0X&{hSGAMNs-CeR*msqWp2vk+n)4rd?}SrwP^PYv%NgTd zxr51N&D8Q0me=|ANt{;+Dy|z(!b^^p;(l11^^Ed)l$=|A^;aG}^E@1}U9LC3gQLM$ z`An#_{~~fi_X%8yEG&>kOhSm|%E>w$k+rKCmbE)7FA*hNr*S;x)sOUL=%6#DR9fRY z*RcO$E9;EsS7CwzmIlIhek%sT&WhIBcp_QjX*0pc4|9^BK>%C>^<;RzA##ez5<9C*($>NwKKPU4v_KR0o^9JvRAWV-p58X=4)wHeCsuu=Jt>(!eHc?Xn5m z-C`30zeyjPs`L@q)JZlGX*LlZY{GkNa=OMQ@SF6pi2$3fgiVe#n>-zCa`)IoyTv9O z{3eZT@_3c0 zMxJ-;m>2Gxik@Mo_!NYrT;-m%WS5<@XKk6j>v@6;*dY{C4^gUc7aN%eduYDI=B%zw z?NV65ocXJp+Q%enBIGHIqBDdH?SSwr*g3b3Lc}V=L>u*B6wU^dBn|%&c9=!4Htrx6 zsIeG9$70u6nUHKgEaGW-@8V8so&R17F-4wg{tFrIn*Ye8o#($~mtDkvLUi(<0RMsi z!2b^LpGx!JQSbi?85{rKy#oJVvde(~z<=OB@c;6)XsK?Pr`VDVyC+%95*6 z)r5pw$Ml_h@vg9kmU2})Y*Sqk57g{DHIt6;w%uYs!&RJpB7_t&iljIy-cZ$gR?(>X zQXW9

    3WvpyqF&=Ieo)e-_kyfLb$cbK6Oh zQVF#}!U}5QoffDawW+QMKVO*EK#l5w8a)eY6rk2j+w6AYc~nBJkg$T9c&7zwM{TMr0#G|s&7?O9 z2+{zxLP!ZU@kR&Ksv1>a3ZQnbwvl$?fRsWFZ=r@YK@FdV8V0CclnP-x?c_pZ<(bg7 z>2EWkiHW+N35_8S0`edr4+8Qa-Y^f&cH2pW#>&&6OZczTpgRqJsSdjc{s^grKd>HH z53KJ3>%Df8p0VO1wg?uOVN1nQyBadwF8jd`5;mA`k05Agp1Mo&i9%1>w&*Eo0 zt&0OV&RyZB)Wx@VmT6OBQ>SqM5yDjX&Q?dx7BMDbw`{s4b~|C_^KJ~71Cu;J!!Hr$AH4I6GgY`B2|00RI9K-vJTK1XcH)5n|D2k~c| QvgzZ^e}s1x_UWns01%r;Gynhq literal 0 HcmV?d00001 diff --git a/_images/sources/rate_limiter/token_bucket.dia b/_images/sources/rate_limiter/token_bucket.dia new file mode 100644 index 0000000000000000000000000000000000000000..16761971337dbf6890ce0d2255b6d7c75bb125ca GIT binary patch literal 2752 zcmY+Gc{r47AIE1A<20DYiENpWNjb%sv4pHKI@XjD(#RxCwqjl;jAR|tv6RWKWF0Y4 zwhWF$gpoaa_MFCGQ1%y{^Iqq@|9P(K{@u_0``(}L_qne}ngRjr4G@@}=5TlX!{Knw z%oWR4_7S?yGNyFZCca=&{t2PSw8#PETv!+z30qYc-5BiBhwwn4%Y0aWZA_`^;JksF zF(<1laDA5M7;HW<;{LQP3++Ff5Yod5Yd#@m+&uYgr&v0e$z88K7qpI-JmC0#)I`LT zG_Dr!zu0B4bDZVk$l;8RjPNdQXnp7+z>Fz>YE|#ZMox@fM^|xlg63A{q+&v~PF%NR zick8B2#EUk|Ng-BM~5_T)+o}p3fs6<(o-;eg4$L1UA#ePi^ZZBZ*LW3ZM;l>w0!QM z{1>Hzp`$`XkEk=FwS`>Ur))p1v}`UG-8||-G80_3-n4Tsb;F2K&-9(pOl z7a&$k*)fu3W;ENO+a1$`j&7J10^i^Ts{c}15O2%ENvcwhUGQ-&FQ^K>(zR72t7!Xb z&DW&q6iE}3^-6T$Ss;ImJmt1zyUxN_wklshfGy_j&g~K6pow{#dNON#Z0iTiPeN+; z6m|)VwN(;oNDb`kJZa?v>>*p> zzvy_Wipq22Z4vVxopPQfHq|2}k?;JA8wNY&(oBvXinAsAp=9~xXvz#HY$8&r@Z02> zuQHVS!uc;VwFxw(K*I-_zN zfHKL!aht^DDX#CB@0AY|I$v;$9`#Mx4KUBCdWHpheR^^xQU6{g>X0jb33?*oZMJF9 z;NvL0q!@(+KKxub%E}h+WLS$*rWcvUl>MFrnNMm}unyJKPr8U>DHbtWfQ9ZQ}?W-EXztVVwU#%Y(f8zW(o#e1M$?~fJ=CB6gw2RsbGj-A4%?z2O*FuguDn0w zkBk+I)fJj6)$xJr07?L81(3I*lX5r_h8l*#j8db4&SP&ffF-58yky`Fz>&;{z;ra2 z0QM#;l{^Bco^JTRQjIt-Oc;vrpEvIFBPXFJC$_x z+=}YQ+#qD4g|=db%)Q1X1$`IxGw$en7aQv!`o*BHD!1?X&o+HZ)kVhqs#@bstZPsk z!IwqXD_Y=CBd1Q?ZOIg$53{d)Mxo%*gMp%EQ~5>3snOL`dcNZX;mJ{J&YjiH!CAFM zvqK`v!y~;)c?K_sC=uH`Gj~o3MpnG%%UZi1dCJu5ZQkR_C{|yI@7uxcJVA`l7q{HX zId{QwtKO#nM3~uq7sGyWBj3D3583D|S&7NxnpI)cEA@?S z5D6WVe)WfJEl=5Be1umHrZ7cR2V)tZb0H}LIL&uGdC3FXRbg!}AGTp+3G*iP)B64G zT-BT#GgoUF{<}J${3*ofJ#{MfRC@DLw;~}dE}IeMX$rCH$`pL_fRQ$Tr!$2wbyl>j z-bjHkt|?pJt5je>MG%lx$yxHom&zHhF7!VY!`(#kON25Ul2?r#Fg*!Rryc0{6*>8B z+bb61;NJB)6BsE@}K(%nsYq0`XFo+C5cpZdgx~=luGX{Os4ne`MviV(vq8!LrQ-nmt@=*>(l}{x%?n(~xu3StE-+QYwe}g+@EK)!Vb*SnJc=43=C%Y9 zTPJHYkyACA9|Cpg;%axh|C2K9HneJk@hHF3QxQdV8_Fs0*34@hu5=q}y=+p` z+&c;Yk{3cQ<^@&B0&Cjew@E(B>2{NF(TbKMak_W2|&y+*;6r0ae>hR|@|-wgnALm&bbjU!7!w1LD@d(0($}!A89X^HFx4;7k>O_Vo-F$wDR;n=Ym*4I)l#2VO zl>Pu|W6g%ajIfuB;}S_Ab~yRpFqWfm$6D59fjDP}*EHW_@Yu9}d_4|h{Dy&bR3k0* zZ&V^J^)iP^pxxVQFGB!CCNIcP?+_9EQSW}Kq%(0>?M)5HJ}o;k;4&CE90zcu#TV@Z z&U%dA&%;k14Tf^Mb8!H+t{nO9PaO6Vpe5FFsTlz}m_o(n3QN3GRSgIJ4+bKWr14>d z0M$(EKT`He8M-H+?l;{!D=^SW298+4lKl|yIv?22%!QNb)6Y(7h?ac-0LZTFUF>2QFQ=|r-nO`hk=6rWS sZe+aRi%1#x{%58u2%2JfFC4w~M@EZu!`^?Rw9b|g99k(euLlJD7hTW&ssI20 literal 0 HcmV?d00001 diff --git a/rate_limiter.rst b/rate_limiter.rst index 63e073a1e92..1d54faba331 100644 --- a/rate_limiter.rst +++ b/rate_limiter.rst @@ -28,22 +28,44 @@ Fixed Window Rate Limiter ~~~~~~~~~~~~~~~~~~~~~~~~~ This is the simplest technique and it's based on setting a limit for a given -interval of time. For example: 5,000 requests per hour or 3 login attempts -every 15 minutes. +interval of time (e.g. 5,000 requests per hour or 3 login attempts every 15 +minutes). + +In the diagram below, the limit is set to "5 tokens per hour". Each window +starts at the first hit (i.e. 10:15, 11:30 and 12:30). As soon as there are +5 hits (the blue squares) in a window, all others will be rejected (red +squares). + +.. raw:: html + + Its main drawback is that resource usage is not evenly distributed in time and -it can overload the server at the window edges. In the previous example, a user -could make the 4,999 requests in the last minute of some hour and another 5,000 -requests during the first minute of the next hour, making 9,999 requests in -total in two minutes and possibly overloading the server. These periods of -excessive usage are called "bursts". +it can overload the server at the window edges. In the previous example, +there are 6 accepted requests between 11:00 and 12:00. + +This is more significant with bigger limits. For instance, with 5,000 requests +per hour, a user could make the 4,999 requests in the last minute of some +hour and another 5,000 requests during the first minute of the next hour, +making 9,999 requests in total in two minutes and possibly overloading the +server. These periods of excessive usage are called "bursts". Sliding Window Rate Limiter ~~~~~~~~~~~~~~~~~~~~~~~~~~~ The sliding window algorithm is an alternative to the fixed window algorithm -designed to reduce bursts. To do that, the rate limit is calculated based on -the current window and the previous window. +designed to reduce bursts. This is the same example as above, but then +using a 1 hour window that slides over the timeline: + +.. raw:: html + + + +As you can see, this removes the edges of the window and would prevent the +6th request at 11:45. + +To achieve this, the rate limit is approximated based on the current window and +the previous window. For example: the limit is 5,000 requests per hour; a user made 4,000 requests the previous hour and 500 requests this hour. 15 minutes in to the current hour @@ -66,6 +88,18 @@ continuously updating budget of resource usage. It roughly works like this: * If the bucket still contains tokens, the event is allowed; otherwise, it's denied; * If the bucket is at full capacity, new tokens are discarded. +The below diagram shows a token bucket of size 4 that is filled with a rate +of 1 token per 15 minutes: + +.. raw:: html + + + +This algorithm handles more complex back-off algorithm to manage bursts. +For instance, it can allow a user to try a password 5 times and then only +allow 1 every 15 minutes (unless the user waits 75 minutes and they will be +allowed 5 tries again). + Installation ------------