From 14016535a5272ee593391a850d9d0676b1521d1b Mon Sep 17 00:00:00 2001
From: Eric Liang <ekhliang@gmail.com>
Date: Mon, 20 Jan 2020 15:22:21 -0800
Subject: [PATCH] [rllib] Add TF and Torch icons to show which are available
 for each algo (#6869)

---
 doc/source/pytorch.png          | Bin 0 -> 12449 bytes
 doc/source/rllib-algorithms.rst |  31 ++++++++++++++++++++-------
 doc/source/rllib-toc.rst        |  36 +++++++++++++++++++-------------
 doc/source/rllib.rst            |   8 ++++++-
 doc/source/tensorflow.png       | Bin 0 -> 2685 bytes
 5 files changed, 51 insertions(+), 24 deletions(-)
 create mode 100644 doc/source/pytorch.png
 create mode 100644 doc/source/tensorflow.png

diff --git a/doc/source/pytorch.png b/doc/source/pytorch.png
new file mode 100644
index 0000000000000000000000000000000000000000..9856f592d1be82d05350237d9849e842561539ce
GIT binary patch
literal 12449
zcmaKSWk8fs)Aqf)bc1xr64KJ$DGSoAba#U^OP6$ugn)>MNQZPNSTxeYQqt0~#PY6B
zz3-3j`S!=&H8W?<Idjg;Tyqk2v{VW4Xz>65AXI}X>j3}=eG39`u+SGXpSKR^3yq(O
ziJ!iwgI|EPk3FDZ=V@aPRdcs?wAZt@whQw9YA+1{5DsSp6F(D84M|&1cV6qiG`xZC
zUTAIrkd_PdvbJ@#_k-HlJ34#FFduaEFhiZ~WSEUbH2F2Xl<b|HVZlE3`oUTTw!yBp
z5_ZgTvQX(jNwffWdp~Pvpu3xguVkPM^FQQDqVNB{=3|EbgT&8OhWVdLnP}=jl{|gy
zp(4DZ4{Zg6MWA95yaJ*^!V(fZP(gkHaXx-=J^`VJ0-}<_{E~u#(0~1y(XIK|IY{a$
zKl)c&=qDLwCqF+gNj|=SfB@bAAzn`(M?L`w2?;)aK|VpjhiHn2zCj*-)`1T_d|Cc2
zLD}Bd*2me)&)L%h`d6a0ji<k#3^Q8Oe|Ev$>px;WeE&5}=mFykwD#f?;N}0@rGFD@
zYX1L8-QE8~?dzv!|G)bFKLz_51bNx>>Dl{w`uo_TC(eQ8Z&O~9N<Q}1ex5!Co}O<1
z9z`7|Pd`szCr>Y^lG5K%lYnw)TH88%{C&y!4=znjNi`2&KWh(Ldo^VlX0#k$XJ<P}
z6=894VSXhsWhF&H0Ra_pMIjLpB{2aJ1#tyM31K1efAcDP+WNcOd-(mE*Y1CL#s4es
zUxRS>Lbt4J@8kT;-tLi)r#ti?9hY?e@4ATmS9$;9wfpb7i2hezKD07?f2a5VO!vQE
zp>62z?SG6H{qP_2xA#EXybs#A>5qIf0pKCAnzDjH;QU?~cA>uEJgVlarnZA%7qJr6
zHb0Ewa}s7>97n@>x2t)#c7Qnr&+g7ndwW?zQc^WXf-wFoSdQc=C~mu}Vw0_P9X|4-
z?s$J$@kNMlR%cod&gFob&!1~)Pl<ydVwyx{8~{kqe`SRWM0~<5UndUe`1L88SQds6
ztzhDu5}C*H=`E|RwYkoraNH+K6H<4ZkwOV&1%*f-&j@{>!ouI*A44tkkX$4IMKJE^
z=>2<zJZg)uy42k5SG%EZDi<xt>e!TrvVZNUH*1RI5gW~QsIFPvQ|$~1NhRM19K1RX
zT1?GJ9SGpZhm>P{(Zi*tx`|N%Bl+w|Ql>x%q|2{c7QMDTCP7*&03;w*Hl)1?34jBn
zGXh{B!Sh%u;s$sNV&z3X$0tELL*)PrsK7ut`l<)uT!VlFfb8oD1PLMm0Z0%)0-!&j
z`PQ)kd0;i^M&}R`0kHBTpGV&O^P3lKgnMuPFtT)j7$B7gZ{Em<1IhsI7LsWXTOQ+|
zd}I&zu#X|~09o3N&OWw0A<!QV0L=er4XqfuO&AEehYdi%3EaoE;A03B42%*3|K|S3
zC(Qr*2HgR4f7a3eQ$X)Y36~%FS6x<x{;WuJC#}}S#>u^Bsjj9Ud>KCUlLr7?<#2o@
z(g%MK^RDps`8%Yq0cj=}2vE;hE|obx@IG;^oz)As*VH-ezZZoAykbR<i?DJy?HA$*
z9Qj{$CDSCxXjc+srBlugy-Fs%=JvSDR|^F^)!t-z0J<mVYeI^AEDQ^0(H!4=lRx?*
z+ogDF`5U3;xPXL1oTyqx!W3=p)9UCI7Snk!U603cc;oT_Bjh_6l)yAvTU5tLb<?CA
z4$;(Y-C!T>6p`CrlBK*QDPYA_NF_<Q{B%U|U0^4e$d@{)(tghKG$|7v_WbTSRhsaf
zf7o+RF%ueekMHaT!qZx$;oh0`%iNBRU9yYT^rPh>$BA-1THqsnuzW_$_Nwsr(-XZP
z`kTfrUHl>NTg#g=p2*Kd60dNmTh}WkeNn^nOm`x4=lI6O`fcIq!~9DRU>JNQX1Qs1
z1ys1PB#B!GyDOnK&fa}gM7b?dKO+GE3D3Ccd?S6At+5b<AGgl)Q`fHHPIv|fUm?VV
zS%=kYyOQNZPL<gT*oA-+c#$H;hm-e_*QOt%#IPOV$>HB)H-u{U*S1zV4Nii#98p<O
zj~qxM-e&fgxH5##Gs`{B{Pf+?@piILa3p;(j4p(i9M`PKzC=VwY@4f28Euw5STb^h
zQ#=b5o0a)|=Sjaz8wd2r((PB%y7V^ZO$s2-F+e(En=?A`CC!Zk+P>>U3n2mI;&&ve
z9Fr6GGMSaN8@n&ix^GzJKmo4Jo2F+4q0;m7n{ufjBcg+iv^5GdcGAS}O*{*PRxi*L
zLE*z9NJNdS=J|P=f5G4jr=%Q;z3tDYiP7$Zx-|5*5PVFKPVNn@>%p(J={IOGk)i`6
z6&?PptN;h?V@~1ikcgZpfP?<r^<AswN#&K^nO`csE(%7hl!=i92=?hI0NG6N`j&HK
z2hq6`3dZL%!d9^i`HrtirreMkInp~xs%;Wwf_E)-juc@yzhgX5B(pY*!Ri`2S?RN3
z0w*lgEg#=Q+B)RNtETUA>dhI++)S#FyBJ`{V%0q*iD)CZw)Ib*ktOcQGm`xVT`XI#
zIgGcq0C42RVsD?a^stK#1-d((dG_x2s1E{?NzC6uS7dVf_>}=D@GJAAc7h%$HuRgI
zhv5U_155XqY*!TfGw}jeBjG^G@BDHE7_aEQq{+a$-}Iw~>!o_{)s$aqsVwZ$mnXld
zhndg@*6-iGYf0;kQUKbcZ%EDqgeVPKx=eQn8CJLch=~Y2tqj_z;X~`nR%b&JA&z^!
zoqut|-v3a&Pxa#Gui?xXjjUgfQTzLS58*Nsr-EbD7*MT#)LqlX;Lk-A->%GVvGnXE
z4l4YYE)oALr!pCypG)nHS6C$+@sWd2K!EYwR0=zAaKD1VSDnr>_}FUTZuQ`6v4SU{
zmG*SN=4Z2<kruBvw)HMC2K4@y<sf6r&Y0Vtm`Wm%k*><u4v%1|O|!H5&$0Zr_@w-V
z>nk1}?!P9qU+alJLO_9|%#*ypOOZq$^BKCKr*hI4@#QR(hOIqHWOiezF+lw3r9AOZ
z`*ahkI&lP-+{jtIiE#X<a`>V#d<K&)&D?8@?j5l6j5r^IwP*0p&g8kjxw=(tJYPwD
z*G?|evjKd4A$Q{x{{lV^;q9xkGZ-`f!FW#$jCJe_e)DzLP0PkvvHkZQ8Sbb3WyBo#
z=$$(4-ku-k<MG`mrwfik;?u=%9QAadRth7Q1h9r`UJlV<WljuXOi+T*v3>#L?^4m7
zpuSkkIx-!|mc_A@aF8>$**GBp(uuvHEm*oPVPpL3A>?E0dQ7AyL$tpjBX0z-3NSzk
zBBRTu(Gckw`4PsDFRQan{%c7_mPPh*a`$^I7b!s4a|D==lTM7qn?%~_@WSh1;gFXt
z<Z~1J@!ng|YU?)bY`!Nyx90FjBOd8GP1uKuO9uos6mOTls9T0u!`}yoYoPi>*T!1v
zl=;asPDjQH?Kzt10q`r$<`%fF*`W$wV>hFo3t@W64#Blp^k$m<afotNxll$Pb4*c%
zeqQpAJ2F-Dx}^mN^S2YnZG<n*d?UPIKF%tD$`MPz-)F+fnR~jjZrH)9IUMajuDt@6
zsF07$1#u!Mlc-tMe$W-lcq%3#71N+9w<jj(K5L<(a)xB?M*Gs}H9Vh(_hJwD$|<C(
z5`#(Xr5EJZMvWa899!bcwNW3&f61d}HJGGidWQKwozNM-uvPWd%5+|R8QplNlEsQ6
z-{hsLaC|Cvsu48t&?))`fwaxP`8<0<7l)!mrVYfp{V8y}9@f5ie8)%<!2!hH3|P>o
zOYlV~HaKa=>@RL&9}n>&Pe_px>qzaIz+{F|78(7`#r)kiG7Nc@bsGPX0Q7FGi0n~I
zRxZ;*krD5m47F^E6$>H->eg25Uj%OKy$^_!N)D3;41oJ&N7INs<i|(mx=j~X9R;D&
zpRQR;#EGxt3*Bb#d`Z&rsGYw>(od*<m7K=`zv3_6%Q~M$R^@&#H$Q4c3HRQUhDT-I
ziG#n8o;CVJ>#`oA2QaEld|2>zX>rx4-;k2jTHP{3jDA@C%ahtI-eX4WXRt|M`qs66
zVVnpsY|~_+8q&C&k%)6|xXf|}&~`dVF+aZBk-ggnzkE%s0`|Rf?0nWTtb+p{8@{}b
z_>qUF*<RfCK&X9vlW8x_{g_ynE9%osHVd71r!4>O!_+bkk_eCfWcs0;$%g%&=MGDr
z$i%+`EV^l&dNUL`TAKW<fK7ISd*cGVt3{YWUP`yZ2jv3EEmX>2;rZ+L%GWpBTM+>b
zXA$usZ)9Z_t@rT3UU7k__dhl>q8#JPvoD7xRf-bhn6tXAzH<0~O^a+aNM%dmBm5Go
zsp{NeY%&Sve=zxM3GJ62j^o{Se7S?rZ>PvuP2YBZ30m-~bWV`M>v(iIHnwFq`6&di
zs5Js8#B$>qmifhZdtjnjzH&WWWPQh`jN&Fi%E(vEf?Hm&6N`>Kd9rJf?8%sfb;IEv
z_TG4B5nRqoh=CwKveNwqx#d}WGQM>_u+DcDr{3O=diF}+t?U8MRqb`X&EUMlvWqFX
zG7y0KX6r?E&#5LH3(XgiP~z)Jyt(J3ao%?^_4CLEdAgsI$8szNNwJ}`&?OpPwD8~e
z2=J<YVYIv`Q&fos7vuaX2*yCL9#wWbYtK>7zd2W63>^Vf==a6kstX%thK)gCs}TQA
z#776$4a4S69S5DDknZnMRqWSqx?WYnNFtW|#VdnmDQH^g!nmkB4)+3bSu3OhW`4}h
z?(l);KT&<VDQ%}&sD5+Yu;*aN`(v#UT%M`1rpwAtfq|HjB?#t)?_`||U*#A{T@Kye
zj%JI>o~os|J{Wr-8yS0xl4%z66{gW$IP`w#`mA=U2P!og3EY5rwHmeP4}{IhbtEo^
z3|HYZ?UX-crY`ZuSi9@%-i2*HR(*N*{+TX6t7?KQ<%c`XpwyT0K=|}^)+Z$yeH3eT
z=f2}>&QQzSKiBFm2RV_9xRG)UPJzLijh+(4o3Eacduf;YiY9>xpuk<pMXLsFmK!aj
zj`spFu^+*A(&eHK7Y{e0M9`~8HbBUl)U%(96C`Bs{JAK9A6}n}?&3STaNv!_09FTg
z=y}{r__oAY@+SoCoRnTJ++0UcCdLZF-)=`{&YPZ*)95y`CX`2X;1?2`J}H2JF27!<
z(d-HrRlm6ya_{D*>w0X->G}fO52g`nv5C6cPio1l;hKg>mI2SZw+892e?kHCwrVI+
z;imfO5~C-iIi}+G^<$SmIc3rY<Rs=lmMrf+f%xDN`t@oFs?7Y{d`RLgne7-VAk@xB
zF)66Q^7FSI+@QOU6)9=i$bn}0$WL*P)4?!|KV@o)M%)y=stee2gwgDkrX$*cptc8c
z+gcEndKaYi;$~0F+{GE%F)l++u&sD%FW%n;r0hNv(FftGfLhldEa#1Z)9tOYc(q$K
zFRHVz*KBNINr$kbJE-Cou{bF0z%#J3n2OdU=NYC~m+SjUzrH#SV#36CfX4R*%c2nh
zt>!NY)u;ob=386S5Co7fjZikg9C*mb;Ec*rt9S~u1VS|H#eR#2=cR)qvU59S7f>DP
z>RI7+4KrA=?VChmGWfrVE+uZNJN^6a2Y6LJ#Z*{43VsrI7F?Ms%3cU0s|ym+I0+GK
z8sgp6H}vRD3${{@TtLDlc&IkBMP)`Zi|)l{$3-wNOPBNRX8J9t;IacV_6v?%uuOx2
zyETtmLHO3;P0}>d>UV_Yz^4BnLv$kGO2DN83e3K2o;!a@rVi}!xaViPeB^>wuibk0
z$l)CnyHl~=*PuUi?G0n%V{i|rZB48u#l*+}$k}N|ckQPg*^`075*Hd{tBDOfuJt>H
zrDZCvu*x!b(tI(5-JyNg=Jl{U%JXLe*p5}4C|A_tg>#3rH3s6CU{|<#9z{^JbZ5VQ
zt7J7Fk%jl2l+#jn{zke&?;()Z*~2<4_>dM7Z9o-4&R|QM=hkkLVqMa9Y<Z8|Wx3Yl
z4(A807njL6;)(w1Z}?O-38tSP);vM^)swc)HmV5UOwIBJ0=L{({Y!Eqw{lhP@D2&q
zehqkURq)1_n{U%_FRnKGJ-nmsUbU)R{51=JSEEtGsJt`E@g_)_v+h-cJ1AcW;dkhq
z;;Lp3HLH0P0R0eORN~Uj^8quWoYTRt!CMa&yMvZJBX(^}e;C8nmkp|no~~QBG&(@`
zzEz}+pj*XSAlS6?ar0Tk>jk)HD&y6bO}9f(Us9>mc;0)&hrxSkR-$*#V-?gWvx&f7
zD*`H`tVLOF^fawt{%L30w5e<($?GAe1zq@CH~;cYDSqqT7gm+rUS9FsZMwDnVPJBf
zr)x#O)x5cZdxgIfUG{F*4(}G&4jg55<7(u3X+X=GADc#QGidXI5mT3ic9Y^Fp5#WF
zK&F2(r~(-HiF2nbOM!enc*~j3_ev@P!z$=8>~sQelFYY!?CtKt<onS?DbSYt_=V=A
z(40Es?dKx6YG-M*T+4Mp<Tkh+vJvTZi{6?cmFUd-@fu~WC_xiV+~*=7*^gRv%US(s
zK(5W(Hmut7K@ymi-&{Vx+CS?8|51vOjB@%a>zaf&JD{{5XHzi=Z{T(`o!c@8vEgrN
zh5p>SrCb(%--Lz0k_go(PNo9&R#uO_Oz(QQD6g<`3n#ZCIa4S_4Yl+><9{L_P&P#;
zX|%I(u|uu}IXpsymVfXRe7vx#2q>>-2>IDP{Sdj&#6b^2s6YJnQ?xZsg)yJ!Rmi?L
z+1v!BN#&`gV%wwNQ{$X6!|i2lB*5O;wMm=Bv)q<1cvJvQCTq9nQ4PcwlOwWuxZ7Vm
z#ZV8-uXG4&02%#xkEZUly5%vGGn3cNL|v95AoL67k;7K+fCqI$_ZBxOS!*Z!fjxF~
zo(b}10s?D-O;2xgWN+dz-xO2r5C<y8_ZooIUr9oRvyn>iCKbED7sGSm-RsYtfM1F%
zTaw1rlKR2j+HQC1rQ)>?>Hu3GlOv}D4H8KC;dN-5;rtNU!IQ|edu>7qbV96bIYqN<
zPbjDWa?sw{;n)pChy1~xUZRS$#<d!^%G43qPkpV5gAcE>QOs4rqMT1uE7kBSfy<$T
zi^GmSWsv%6gjf@C>mC;;NU*^{NYk`YTvUIXTZkeJn2=x__@tAY5cNR*hClGs0lQ0j
z{SHvk{f*D)i<@csRT28^O0s~pfv~+WA5EQ+w9T1m(11@ON00_>p23K9;n$7Gbi#Z7
zaG0qyPWQ6+KAXdv$YI@fe+yku?=|WLeOA@!_M^D6gu!&38FI3fX~u{$#Dgav@Ng*9
zOYnO?jjraq3Fcw~@Jk-Z%c@fxFRU(I2%k~;q6XGIOpX*<3m)SuC;rrYU5xaqrZEm)
zz$i{jpC!3!BbJL_GlKRlQ5-j?A`%DiW>;}8)inU>g6+MM^|vXl3T>v26bbX$<O!!R
z2Sn7RM5<0S<TEEAY81EIm*FC*Z|EDih*jQVPhz}f20CY1xw<J5h+B0<<5;)zDS-DD
zRl$SrPK~foNB4DIKxiUREk`rwuh}?>tk>n5qPf3qjDYrU06KF9<dWmkx4>wU$lUbx
zl?@mycD#r_lO}9$8h~Xg@M&!uCiy)AdEHXUQ$S7ep)d7ASB2F}B^Xo&KL7|{fFs#D
z#|<*bMW1c=F_J3EssIofqooWoHo+Umivt4b#(wBD>8tMANMKrxJvU?m_lnSfH155^
zS|4FE7TF^_VSj^(2rT#@tQ>_#|KDP%sf4iDnga%E-`gz!VAd3j2UO|3Mt`B1ivkK2
zKYWVN1RFGoQDPukALua~a^<FbAIk#*pvAJ<wt?+cZE*+8-iO~5JD|Ag#M=B5<T(yj
z1Xg)~2x4lyFs0&5z1X|2@F=limNu810#GsJ;ZcoAr(em@%Kbv5#x{)B-mu>xQy3cM
zqiziP3}YqFU`Z0C*TuU##0ln<t)LRKd%Z<1^><oWx1FBf57^>_&pgoPT=ahX7QkSA
z;q<4?*!&|kbRs8x?#Vpv8(bZTQLYa{;8gvX(HE;?G(3&tNPru-RKBMUn$OV!9Y(`8
zXZ`+K0&J$ynd_I|2I+bNbt5=Jds`&#lTn2u_}R#PyT)E&kPDEuqsLO?PWXIZfitcW
zyaplMx31USc7S|T%6Vb-SHu^@thC7B;zx{+Y6%<lh|BUBtk04aA#dtn4m7j$*?9_L
zP<iilIzP;$(Oq__O0z^C0(|+Q_uc6_m8Tl=g)XmmUc6^zAd~_DDt%D)=t9lz*EdAK
zZ!LF?WqkR7<XuXa0iVr}VW0JY*O||9wMInc8Gu(j>E&wQ5c%}FE4ZhZmE}0El9Js)
z(4$TiRuAiSHhuY`H@<+9Zr6{iC95((WdL5zyF4POmKk+3G@>W3RU^q^Fv&{u9W0M9
zkH~GQn|;0RGzpubqOlVrz(6mcM3B)3IaTfs%!Hx}?d-5-j$oreuC`+lpcNXs8&sNU
zbQyQg3|suW3pD{7o(>sgKdw}Lb7S^N3!aUVKFr0zMgrU`Ck_3k{6a3&WScLl*6ok|
zG@(v#vLMtuGHe#06Eo0h6HI%V{0^e1BA<%WG_kC<tupY~a0j13M;zFq#>?-kmq2S@
z{CDS!*!w;=@b-WzH|HT>;}|qWCHXe#E$1y({sp%5O*nlScNK^sR=%L@k$5sop8+T-
z=1z2A?rZQM6L%M8LPEpx_k%=ykX6R6+$ui9Ij)b+w%EWhjcumpW>gh88)&G(4LnYs
za0I-PyuZ-RQ&EA(GDc%wx3{P3eDGu9kD>I)9}piT@(1KwH}6;lMw|<%jb?Ax+~xA1
z&5Y-<sh?s21alk-6KghB_7*uW#;T-zYUD7(4nGk%3)7E1=|}rsH>HOcY~`jzg<Rq{
zao+Tajce!>&<^Vvpjdq8he5KfF2SoIVoC7V(P?N1lsspCTt0!6d#tORDAi`J$K9ao
z3JPcgo8~OZZfij7Y&s{UX^5~u?f0_uc#OHifVY#@WS^cHTiqk|1lP?BA9&&=5++?h
zV5DU|IdtM<4qL<y_*+DFKJR@EGQF8=`{zQ@xKnE@b}QN|Lytp<1Cb2Dv6DbyDUDN7
zae;jYo(9zzLw=!07c|9ozpS4W(6Z!)tPI45l^(a<1Cj}AQ2qoJp>`vW1-+fy&uAf<
z{q{@|LW71+TIiH1`r$I)?pm9%MKK;cNqP*r?4E1oWmRQXe+(?W+eSwXj0eJL;1kS3
zTy#iwlfiLAtln_!8JBQm{``PWq^IN^xNd=A(U<`zubyiU3&AHUGbDhoLG8uXCb}T|
z>)Tl2#My2tXRq~mN7c_BP7lmp?r$ra?x<$^ikpSijCHEYCm*<kw<mg&L6s10rR@*j
zTbAF)T?{qX%iIrl4T3I`w1Oj!{Ifhi($e|0S-pI3>n64??hwHV(ZbHNH{sj1p1vTc
zt_!PC0@t<7wJI7RC%k`RB03zFiO)J;T}o`O>Gmd@4KYy0O+-sTv;-K~J@~wq_}=xx
zG!%O{@T+O1tEW)K@9+WU9#`AyJ5xwaCt$U~E4Up}#sRcofGT6~CwV@zFbv}E#N3hB
z&)i$A9~#aZ>;kNKsxLy-tES$@6XX1Pf(WJFFd@$uLD+d)mY$H3=VZAkuLrLa7>fkU
z8QWIt-UHXYnd^8X$|3wN11lyy>Y{Zg-t2{S2R%Ca6{LP)M=ve=IaEa+^8ns?eegYa
zG}9?`<}HpqgfDnn_&S%yd1Lv$>$?clnaRNv`ws$@=paZq_XWHxVE&$D((9ibU%V3_
z=&*%ReX*|`mj$3MnQEt0SB}wPA?+=SjCPM>67r#+iA;oq6H>i?GNMq+eOvba+x1Fu
z{^#aDu^Aj;%WrXe+u07@L>G@)FjQzg>`hZftQ1#iv+Yu|<VzqnBlIWOuU-!;6kFyj
zmTb3)DLFmS+kE<nwE#HuQGTj-5I!SceL{MeDqpH*d@7%D63*}6JGwGxHuMvna^IW!
zax374ccYo*=`F+8f=+>~c;361qRSZ4P3lr$3-ak-rdv0*aBgfdTnLI-?<^U4B<nni
zWylO4+UtQS7h8^BJrnK;Fu+9Q%bDGwqfeo7bzR{8mX$TfLSZ-!Jl2^h;@(kD*Q5g#
zwZBY1SM#br`Q1E3&dmjX4`u@D4eTg8D@n+2cp7tB%3<ZxzEQC2OP?1x2jL%NeTns}
zpyV`N)nK38Lv{Bc>RZdqYgi1-3B3+Jx7sRI!)u=E%D-+z$^Y2SL%FaG_4}{l64?cu
zE@Rnsr%vc}Jq^3OL}*v#$=#^c?0$EG-@U~LEKY+cYT)Gt$zQ$m^IWGk<lshw&(R6)
z;NDVI8c_hXYL(&TKyU29#k+Y+m2oGI6->ZZt>}6ejf9IfKf~acj#4(u+i$aX@E|L^
z2h%BhX><P^ozIuZlm)L3U86QoUi%9a?)c4hSOp}2!r;%bmd!J+IFR`Kz`Yn<^20}#
z_<MM)w0ft>$mYl=2YL*pJ;6b3xDp>#L_WZq;#7!ndT~t*CcFK5@|HhWae&1sU=!n2
zCs>Yry&s`t%-)APz3+|D0W?a1LJ7Wj8O?4x%Iz7Z&%OJK1|*M6KQ(in)R#=syADen
z|N16zv?xb_$|-|7u+H&Y(Tu~K<3~)jpBExR<VW-Mxm49r)>JsXqljgZ4}2{j!7r1W
z+8Ocb`hQ`LsCsh4iiu7WJk7ElXihYiPkTZPe_RXr6|wYiTgts*IH;8Xk+5WZS)3Rg
z@}_*{P);-gcvD<@I@*s&@qLyCJCT)aIj$W4CjN?~mhtSGzdo3)>hfzsZnxBJq0tPA
z&?g5^E)2^9IEnHMC>gWMN#9(=ZSW4pf&CKsUhmNCmXQXPNcd|Q!8ZCr0s~xIrP^G>
zRTi$Fno?<MeGNimCVQ{QSDbUM{k@}gl2X4S?d+B^-5|Tb{3q;mA!|y)(d)N8DsoeN
zRdiRj_Xa=rR1%BG#+7M+(2!Q$0EP5vxZy%I<)hnvY0-39N{>VAfn!ScdSQBoRByy$
z$P%KkbyQQpaz|I<;DDU5z~t8!(hY3_bPqqIIc%=W=+BF{qCNf&$64Awf~~Y(Yx2|Q
z&q&4x+fI~D<Q?ml{HzzvyAAt7=3K@%0*9Q&>pG!nj81_D)vm9ffSeF2V(OUp2a)>1
zT7OoGB9F9AD4wfE;e6sj#{ce=mYE6H7XFfEZO^g#v*CB;<HbH(C{P)RHb%<wL$$Lg
z1U#`<m=*jy9jEsfSO9!3j%;M;@g!D_zZo3qUml?%7P~e7?pomt>is;AeK937#tU=j
zQF)|~@rIeyBw4wa;B=yMlYN;X^W^s_y)ud;DBSIxph5!ipPG&ZmT%T15km4+d<JKE
z(J90)x3<Czc))9gZ@yNKaGIfE`wH-I6K;H}XsD<lI%)EuZ&m$%5)g+HMgu6{@plWt
zOq;tM1iYUN(rwh>8;J$?)HRH|&RpOK3QT!Ww;mVvHxDoj%F2tmm&U8*wS*e1Bt4$2
znQ4pN%3zE~h3IVEO_dH-+F1XxQ3MX1Yp~C!mQslIw%%?Lp7bxk&}jYLMmb*h%S>k}
zMyE=`WG&I6E*HNve-`>UUvphnxj&|)ETkxujDKq&y5a#IOA%ki|0&bZ`iCgAIZ${L
zxsq!wq*M_mF73I&7*eqWIepISF``+}PJru^8o9rl(l2|L*c-+@nk#zijV`tZynk=0
z@hk7b+W)!%p3{0WJoxBho!8?5#X61@8bo%5;Vkt#eveGr>6h1^LYFROMLc2WHslnN
zQF?w~(<Uc32#ZHcq!v%rd1`EbnCByByRYU<7r=C7%_4#Hm;8wxw(rt!r<-j<LdDQ9
z1h~ln`?kgmwKE>e4PR#2z@^W?_3^`Y_3;~0>#+?u`0!C?UP%^>pxk8gEVfi%J<8q{
zdDM76B9c*rGLktfd@D&ecGvMjtD}U;8qt>d7m=G6u?=>+pJUfa;(4O-;#zq#h$6Gc
z)p(dtM4?n<dRmCB_Q%rZ3iqES^N@RFK;i_j>~)#AHzr(ZdBts+!f7V^$zu!GVBdng
zz|XlP^@ru*Z$x35CPB`h=RV!b3fCVx?-C(r7w})}zsR2Ns(*f`!1KXB?Dn*DF#fQ(
znlhA^=++A}*sWoE!gKI+_DZg6@;-l``zJ?_S6eGfy6K?<0k+S>3p1TN=AKWBqpQ*0
zv${)iJq`OgJq*s_+zvBhxhW9bUT*yA`Q^IZrU9Y77bcm0(vxW1e0PX44E`j}pMLD}
zl%H+R?_8`rgeN3S=Nwy2Qj&^sVnRJG<$1X4%P?+6j;b(@H<+T+DeJ|V1=lm8NTWv~
zjtJ!h^Kp4OZn3LLQsi=&H1k0bIzGqNxmr*6oT}&&;JA)8EXQw6_AcNZ@XxJ#U$jcg
zE1wsP%zW8m{EH;r4i^L3e&4v$S@kDj$YLME_!7<K$9W()V^VOxv764g2@=%s@k^Uf
z#Ht)|Womudm+h65wdc>4EYfdq)T;@?3D>XK!6jPmh|fE_t%h1v8<@uLYb7X%<Mips
z%_dt7&7V+_r_wsPhdtP9JuLDiP@C7`m7=4$`0(*osF+81TQKst2)0RmUEZY!?+H>I
z|1j4ziE^>u(rCrvy166l+>OEL3j3Y<j{Bkwc^qgGB3Op#wpUVujCTZh_OIz~62(M{
zDlOMOq6&AluhOuX_#Mu(#3oz{t9ZaF3b)t{_q0&iUFb0Dn^0yA=WCP=F|wHWL5E)J
z_B6B7^a8Q=?y_Q`v=kq6dz)%<vd|IKbk$k+LaQfx91#N^*sGxp`cn{NW6^S(bAl$b
zc_BNp@-yOIs;nB`d)ITmkk$rwl$hp{e|@ojaWaq^Q1EEv$KwKj-GYrBJBF;+u@1${
zS3ka7#~;j_3;8S9FZhTo(_tWly{zBaucLiJbvllC6Azd{f_g3}ieL?mr|f<Yz9<`Y
z`Sm!UBU*Je@v>|9>OHrhd!F{!CQU{=p=4`V=m%y+r!HTjEXE-C(Vt%;Gtgr`-RoNg
zK+70og2xu^D@@U4R)idJ8TZh7I(485XIqU-Tey9+<5%O9rCQ=ZTcKqh{%t|_3X_qd
zLhCLUbkuK?nOeU_QgYx{n3U*L@%>~}e*I3Ewgl~7ekfb9l118M6{x42TW9E};2l<}
z76!CqaMmpk+_j;x@t?#GyZt*B`?}n%IbVGqb+>l<H7EMvVAgK(GLiE){%8x$BZ;$Q
zNJx+@R_ASEug#B4yTg%RB4|_1d$j9|t{*Y6i4R9FG30jen2)Yw*uXHf$RRRh5V!Q%
zkZG*g<FNbT*S|&Y-k5(tv<!6XIBV)Sks7j5rJ4v5zAvv-vlJd1B&5l4qm>zRpw6(h
z3SsJd;H{O$QfnebA05f}mhc-XG12v%`Pvm%K?x?h%pXC}IOE82lDAGf7cMJsu8dHg
z5HL_RnI|o_$t=YwZRR53RZ^_$A@4#%XsU+{erKJt5zLrW>g<Y+$1$1&pV{i0WKwJ=
zkCWXzGSrqE(^QsQ^Iq)6qZaXF&zvux256jmZ3;LuD}LfV6cud9h@)BHt}H!uwQZ71
zn*jHc;6{)I93H+-vEPIXG9P3yJzL9nN==r$XJvzH&ML)=^TjG_w87_V^j;ikpd(0I
zyT6!<rXD|0$<)=v?dZ9W5Lt{U`GdBdz`p4IBj}Z7P?_zs4S1@a7tD-c4r^K?D~2W7
zmVvE6-FwHWvZx|UH_&s_DYLA7NlBhj;giLE6~oZNJyXOZL2}I;ixb<8sr~&(^sJ=B
zkO0X(@HSMY<_1-l5UXR~y5H_kUc$4Sl0>N0=)(B1mttw-W9q#xAq&1K?TVd4UN;3I
zJ7THa-_1#xP%qd~p@lYqpJ9vh*s`;YH8ORmxP8MJ#<SQRvRJCWDZ((*xSYT1XOX{^
z#y=kvH*MR74U#{V=$63DA`?g=ENkm$sS7UOk8~ZLoBs@OY3_YDd!gNgoq94A^4R6>
z@^Go$bZmgwi=5yPW`HoKT#P>I&-gu5i>zOLqs#(V#Ow9$3~fRdIh%5`p%EGu$~dqn
zE43FD8=p#n=dUM18}(uSzUvr-*<|y_#$v%@Mb}Dmvm(RY6-3z(MShYyXvgtwS21D`
zUo<x*sD~>!<OW@Omsf!<1|okSb80iVgc1aVc?&B;j_^9el2ikStDVA-H-tkY%Nuz2
zwN6Dqt_Sb%Z;)04_{Q#4;<a2POV4F6Fd#>m(vl+&VIqysj5I&`8V^VFDy&)?$^$5=
z2ebU)x8ni_=u;$d7cr&s)YrYSvV}`EPKTj_EAb7hv4;DP|5ApNrZA2B?vCtSd(2z-
zJNxU1+r`hz|M-0s=ux7YzH_jwFLm<m;l|AUqgWSfdwD<$@41T^_t33Z(`8N4be_&1
z4%$*t5~1D<x>Q8&^v7jY3-~9wZ%>!vJ#a`O^wY2c$HRs`6vV%A`#{~gz+jeE4x;P_
z4TD=1*BlK<$<G&?x`L0JYAA#<W@;mIKp=6J(4XtSlBK`K{5{w5o9wl^o1O4h#P9wv
zh14clc$@Qi{y`c!-#ceZ5H|!o=8kwEalD%GyLBiO?ziPS^t4!osbUE4Eb{F=#`vq?
z0y6YDlhK!l1LM|>7c+7*l(|uLl29P&l^}^nSI_Hn_{eRk+ibkyzGpov&0tZ?E65HR
zBeBuvT0_F6&v|Z=Mxkrx@1zcMy#I2&v&!QRv_t!e1->~0*-(|}O&J-LxSB6eCT5+b
zHQQBBw=;O8&rQ9t8)wfII$_r`DVl^p9(y_bO}((YJnnA_clkXIKMpgtGpTK@Z{oDj
zrhNn{x|c1*#*7Pd%j-5DKi~O=HHu|oP67;Dg*}H82mccN<PjK-b|^K*_VXS;5VIQ6
zfDrkbzE2tEV*7LU)28^(=ft~Rcj%n*{Pz{*vWLkFkK}h`)~qJN>9$r`wcnqa3CRqF
z`=Wg}c=M?OCs?=7Gx&O>-i-CC@&}`2UEA~-s(<I4_;FX_$0BSYkS#sEX2xb^!I0+Y
zvqj-P=Wd=9TfIXVpG7V%P17p~JCUqA@#EqYchpbDnI=B7ImGSYLs+S@1zkQa^;+*j
z{P$g%<u&7C-MrwSxU*GmLAM#A7P>(+BswJ_W6bs%wwk%b{;i(~4HbPmpyI3Y<&#<`
zG>qJk$0M29a$UjWRc|!&Q)@qKP!JuEK%b*q*%kEal20Be0IB&=o}Bt^D%++(F)u=p
zjK*Std9`k*?-$X_BO(HaIU=S{fzLA4!t?JcO<``7b?uxK{?keRJr*573(R4)V4o-3
z%)H$<A&*&$IcT$r-l4Z%q9&^j5XEjmJk}a>{RJY16TleFBLF8rHLUdM=~Lkzbh$7I
zqYZsd?MZoNa{2SD6x-1T+PK%-FG54-u!yE((8n{pq)&y)1RB=9y6{|4Xx|@n*7SwF
zQ(!i3Xnt5{t^X1};9*>ckr+W$ERF*psL;8^O~AwM{mWR?rFv$n=&7i$O7!01tHzo*
z>)V<|fSl>nZzrDsqB$|NdAdg1f27tco|E90a`uNeYR!4IO*G9nfWCjtMJC+yMz-zM
z9BnS`ZdAXn@@3Zpk8o8qSciX$;wO=f!~|xjIxPZGj^M?a?vKMgR6S#iRI7JySa5vr
ztE`h3yuGjDU_Po?-tCwhIXo)JKPQ^2RnOm)QVvs^DLPN+lxt@H)}wnOakK>7+#C)w
zna_mA-jwY<!_|AN1bbV9i$_6$Ivi&~9j^HF`0eGx3@~x~F}2@^FsbS7;YEx)u{sPF
zs9;2UiI#Uj1A>DY)iH^fJ>hlq8BJsvlqHd^(7^@j`JFL=XXFyBl{+-?8E-Om@vK#Q
z<_`ss^bl8N*3GfB(($y(0Ex>~)2K1e%lTI%=<@=M!5>|N`clH(Yc^v=TcOR&d}*F^
zJ|QY<gLz@k5^#eLm`~--_Y?47PO*O+UcY|L@v>XxmB52Pq{v#Y?hX_$`T(yyL4?89
g0oaD|PCEjPcmEXkx#x#IgNFX8sc0#GP_&BrAJ2#r3jhEB

literal 0
HcmV?d00001

diff --git a/doc/source/rllib-algorithms.rst b/doc/source/rllib-algorithms.rst
index 64f2b0eda..2cab15da9 100644
--- a/doc/source/rllib-algorithms.rst
+++ b/doc/source/rllib-algorithms.rst
@@ -6,6 +6,7 @@ High-throughput architectures
 
 Distributed Prioritized Experience Replay (Ape-X)
 -------------------------------------------------
+|tensorflow|
 `[paper] <https://arxiv.org/abs/1803.00933>`__
 `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/dqn/apex.py>`__
 Ape-X variations of DQN, DDPG, and QMIX (`APEX_DQN <https://github.com/ray-project/ray/blob/master/rllib/agents/dqn/apex.py>`__, `APEX_DDPG <https://github.com/ray-project/ray/blob/master/rllib/agents/ddpg/apex.py>`__, `APEX_QMIX <https://github.com/ray-project/ray/blob/master/rllib/agents/qmix/apex.py>`__) use a single GPU learner and many CPU workers for experience collection. Experience collection can scale to hundreds of CPU workers due to the distributed prioritization of experience prior to storage in replay buffers.
@@ -51,7 +52,7 @@ SpaceInvaders  646                               ~300
 
 Importance Weighted Actor-Learner Architecture (IMPALA)
 -------------------------------------------------------
-
+|tensorflow|
 `[paper] <https://arxiv.org/abs/1802.01561>`__
 `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/impala/impala.py>`__
 In IMPALA, a central learner runs SGD in a tight loop while asynchronously pulling sample batches from many actor processes. RLlib's IMPALA implementation uses DeepMind's reference `V-trace code <https://github.com/deepmind/scalable_agent/blob/master/vtrace.py>`__. Note that we do not provide a deep residual network out of the box, but one can be plugged in as a `custom model <rllib-models.html#custom-models-tensorflow>`__. Multiple learner GPUs and experience replay are also supported.
@@ -98,7 +99,7 @@ SpaceInvaders  843                              ~300
 
 Asynchronous Proximal Policy Optimization (APPO)
 ------------------------------------------------
-
+|pytorch| |tensorflow|
 `[paper] <https://arxiv.org/abs/1707.06347>`__
 `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/ppo/appo.py>`__
 We include an asynchronous variant of Proximal Policy Optimization (PPO) based on the IMPALA architecture. This is similar to IMPALA but using a surrogate policy loss with clipping. Compared to synchronous PPO, APPO is more efficient in wall-clock time due to its use of asynchronous sampling. Using a clipped loss also allows for multiple SGD passes, and therefore the potential for better sample efficiency compared to IMPALA. V-trace can also be enabled to correct for off-policy samples.
@@ -123,8 +124,9 @@ Gradient-based
 
 Advantage Actor-Critic (A2C, A3C)
 ---------------------------------
+|pytorch| |tensorflow|
 `[paper] <https://arxiv.org/abs/1602.01783>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/a3c/a3c.py>`__
-RLlib implements A2C and A3C using SyncSamplesOptimizer and AsyncGradientsOptimizer respectively for policy optimization. These algorithms scale to up to 16-32 worker processes depending on the environment. Both a TensorFlow (LSTM), and PyTorch version are available.
+RLlib implements A2C and A3C using SyncSamplesOptimizer and AsyncGradientsOptimizer respectively for policy optimization. These algorithms scale to up to 16-32 worker processes depending on the environment.
 
 A2C also supports microbatching (i.e., gradient accumulation), which can be enabled by setting the ``microbatch_size`` config. Microbatching allows for training with a ``train_batch_size`` much larger than GPU memory. See also the `microbatch optimizer implementation <https://github.com/ray-project/ray/blob/master/rllib/optimizers/microbatch_optimizer.py>`__.
 
@@ -157,6 +159,7 @@ SpaceInvaders  692                       ~600
 
 Deep Deterministic Policy Gradients (DDPG, TD3)
 -----------------------------------------------
+|tensorflow|
 `[paper] <https://arxiv.org/abs/1509.02971>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/ddpg/ddpg.py>`__
 DDPG is implemented similarly to DQN (below). The algorithm can be scaled by increasing the number of workers, switching to AsyncGradientsOptimizer, or using Ape-X. The improvements from `TD3 <https://spinningup.openai.com/en/latest/algorithms/td3.html>`__ are available as ``TD3``.
 
@@ -175,6 +178,7 @@ Tuned examples: `Pendulum-v0 <https://github.com/ray-project/ray/blob/master/rll
 
 Deep Q Networks (DQN, Rainbow, Parametric DQN)
 ----------------------------------------------
+|tensorflow|
 `[paper] <https://arxiv.org/abs/1312.5602>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/dqn/dqn.py>`__
 RLlib DQN is implemented using the SyncReplayOptimizer. The algorithm can be scaled by increasing the number of workers, using the AsyncGradientsOptimizer for async DQN, or using Ape-X. Memory usage is reduced by compressing samples in the replay buffer with LZ4. All of the DQN improvements evaluated in `Rainbow <https://arxiv.org/abs/1710.02298>`__ are available, though not all are enabled by default. See also how to use `parametric-actions in DQN <rllib-models.html#variable-length-parametric-action-spaces>`__.
 
@@ -207,7 +211,8 @@ SpaceInvaders  650                       1001                           1025
 
 Policy Gradients
 ----------------
-`[paper] <https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/pg/pg.py>`__ We include a vanilla policy gradients implementation as an example algorithm in both TensorFlow and PyTorch. This is usually outperformed by PPO.
+|pytorch| |tensorflow|
+`[paper] <https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/pg/pg.py>`__ We include a vanilla policy gradients implementation as an example algorithm.
 
 .. figure:: a2c-arch.svg
 
@@ -224,6 +229,7 @@ Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rll
 
 Proximal Policy Optimization (PPO)
 ----------------------------------
+|pytorch| |tensorflow|
 `[paper] <https://arxiv.org/abs/1707.06347>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/ppo/ppo.py>`__
 PPO's clipped objective supports multiple SGD passes over the same batch of experiences. RLlib's multi-GPU optimizer pins that data in GPU memory to avoid unnecessary transfers from host memory, substantially improving performance over a naive implementation. RLlib's PPO scales out using multiple workers for experience collection, and also with multiple GPUs for SGD.
 
@@ -268,6 +274,7 @@ HalfCheetah    9664                       ~7700
 
 Soft Actor Critic (SAC)
 ------------------------
+|tensorflow|
 `[paper] <https://arxiv.org/pdf/1801.01290>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/sac/sac.py>`__
 
 .. figure:: dqn-arch.svg
@@ -298,6 +305,7 @@ Derivative-free
 
 Augmented Random Search (ARS)
 -----------------------------
+|tensorflow|
 `[paper] <https://arxiv.org/abs/1803.07055>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/ars/ars.py>`__
 ARS is a random search method for training linear policies for continuous control problems. Code here is adapted from https://github.com/modestyachts/ARS to integrate with RLlib APIs.
 
@@ -312,6 +320,7 @@ Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rll
 
 Evolution Strategies
 --------------------
+|tensorflow|
 `[paper] <https://arxiv.org/abs/1703.03864>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/es/es.py>`__
 Code here is adapted from https://github.com/openai/evolution-strategies-starter to execute in the distributed setting with Ray.
 
@@ -333,10 +342,9 @@ Tuned examples: `Humanoid-v1 <https://github.com/ray-project/ray/blob/master/rll
 
 QMIX Monotonic Value Factorisation (QMIX, VDN, IQN)
 ---------------------------------------------------
+|pytorch|
 `[paper] <https://arxiv.org/abs/1803.11485>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/qmix/qmix.py>`__ Q-Mix is a specialized multi-agent algorithm. Code here is adapted from https://github.com/oxwhirl/pymarl_alpha  to integrate with RLlib multi-agent APIs. To use Q-Mix, you must specify an agent `grouping <rllib-env.html#grouping-agents>`__ in the environment (see the `two-step game example <https://github.com/ray-project/ray/blob/master/rllib/examples/twostep_game.py>`__). Currently, all agents in the group must be homogeneous. The algorithm can be scaled by increasing the number of workers or using Ape-X.
 
-Q-Mix is implemented in `PyTorch <https://github.com/ray-project/ray/blob/master/rllib/agents/qmix/qmix_policy.py>`__ and is currently *experimental*.
-
 Tuned examples: `Two-step game <https://github.com/ray-project/ray/blob/master/rllib/examples/twostep_game.py>`__
 
 **QMIX-specific configs** (see also `common configs <rllib-training.html#common-parameters>`__):
@@ -348,6 +356,7 @@ Tuned examples: `Two-step game <https://github.com/ray-project/ray/blob/master/r
 
 Multi-Agent Deep Deterministic Policy Gradient (contrib/MADDPG)
 ---------------------------------------------------------------
+|tensorflow|
 `[paper] <https://arxiv.org/abs/1706.02275>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/contrib/maddpg/maddpg.py>`__ MADDPG is a specialized multi-agent algorithm. Code here is adapted from https://github.com/openai/maddpg to integrate with RLlib multi-agent APIs. Please check `justinkterry/maddpg-rllib <https://github.com/justinkterry/maddpg-rllib>`__ for examples and more information.
 
 **MADDPG-specific configs** (see also `common configs <rllib-training.html#common-parameters>`__):
@@ -361,7 +370,7 @@ Tuned examples: `Multi-Agent Particle Environment <https://github.com/wsjeon/mad
 
 Advantage Re-Weighted Imitation Learning (MARWIL)
 -------------------------------------------------
-
+|tensorflow|
 `[paper] <http://papers.nips.cc/paper/7866-exponentially-weighted-imitation-learning-for-batched-historical-data>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/marwil/marwil.py>`__ MARWIL is a hybrid imitation learning and policy gradient algorithm suitable for training on batched historical data. When the ``beta`` hyperparameter is set to zero, the MARWIL objective reduces to vanilla imitation learning. MARWIL requires the `offline datasets API <rllib-offline.html>`__ to be used.
 
 Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rllib/tuned_examples/cartpole-marwil.yaml>`__
@@ -375,7 +384,7 @@ Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rll
 
 Single-Player Alpha Zero (contrib/AlphaZero)
 --------------------------------------------
-
+|pytorch|
 `[paper] <https://arxiv.org/abs/1712.01815>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/contrib/alpha_zero>`__ AlphaZero is an RL agent originally designed for two-player games. This version adapts it to handle single player games. The code can be used with the SyncSamplesOptimizer as well as with a modified version of the SyncReplayOptimizer, and it scales to any number of workers. It also implements the ranked rewards `(R2) <https://arxiv.org/abs/1807.01672>`__ strategy to enable self-play even in the one-player setting. The code is mainly purposed to be used for combinatorial optimization.
 
 Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rllib/contrib/alpha_zero/examples/train_cartpole.py>`__
@@ -386,3 +395,9 @@ Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rll
    :language: python
    :start-after: __sphinx_doc_begin__
    :end-before: __sphinx_doc_end__
+
+.. |tensorflow| image:: tensorflow.png
+    :width: 24
+
+.. |pytorch| image:: pytorch.png
+    :width: 24
diff --git a/doc/source/rllib-toc.rst b/doc/source/rllib-toc.rst
index 86f74dd3c..b7dc78645 100644
--- a/doc/source/rllib-toc.rst
+++ b/doc/source/rllib-toc.rst
@@ -72,42 +72,42 @@ Algorithms
 
 *  High-throughput architectures
 
-   -  `Distributed Prioritized Experience Replay (Ape-X) <rllib-algorithms.html#distributed-prioritized-experience-replay-ape-x>`__
+   -  |tensorflow| `Distributed Prioritized Experience Replay (Ape-X) <rllib-algorithms.html#distributed-prioritized-experience-replay-ape-x>`__
 
-   -  `Importance Weighted Actor-Learner Architecture (IMPALA) <rllib-algorithms.html#importance-weighted-actor-learner-architecture-impala>`__
+   -  |tensorflow| `Importance Weighted Actor-Learner Architecture (IMPALA) <rllib-algorithms.html#importance-weighted-actor-learner-architecture-impala>`__
 
-   -  `Asynchronous Proximal Policy Optimization (APPO) <rllib-algorithms.html#asynchronous-proximal-policy-optimization-appo>`__
+   -  |pytorch| |tensorflow| `Asynchronous Proximal Policy Optimization (APPO) <rllib-algorithms.html#asynchronous-proximal-policy-optimization-appo>`__
 
-   -  `Single-Player AlphaZero (contrib/AlphaZero) <rllib-algorithms.html#single-player-alpha-zero-contrib-alphazero>`__
+   -  |pytorch| `Single-Player AlphaZero (contrib/AlphaZero) <rllib-algorithms.html#single-player-alpha-zero-contrib-alphazero>`__
 
 *  Gradient-based
 
-   -  `Advantage Actor-Critic (A2C, A3C) <rllib-algorithms.html#advantage-actor-critic-a2c-a3c>`__
+   -  |pytorch| |tensorflow| `Advantage Actor-Critic (A2C, A3C) <rllib-algorithms.html#advantage-actor-critic-a2c-a3c>`__
 
-   -  `Deep Deterministic Policy Gradients (DDPG, TD3) <rllib-algorithms.html#deep-deterministic-policy-gradients-ddpg-td3>`__
+   -  |tensorflow| `Deep Deterministic Policy Gradients (DDPG, TD3) <rllib-algorithms.html#deep-deterministic-policy-gradients-ddpg-td3>`__
 
-   -  `Deep Q Networks (DQN, Rainbow, Parametric DQN) <rllib-algorithms.html#deep-q-networks-dqn-rainbow-parametric-dqn>`__
+   -  |tensorflow| `Deep Q Networks (DQN, Rainbow, Parametric DQN) <rllib-algorithms.html#deep-q-networks-dqn-rainbow-parametric-dqn>`__
 
-   -  `Policy Gradients <rllib-algorithms.html#policy-gradients>`__
+   -  |pytorch| |tensorflow| `Policy Gradients <rllib-algorithms.html#policy-gradients>`__
 
-   -  `Proximal Policy Optimization (PPO) <rllib-algorithms.html#proximal-policy-optimization-ppo>`__
+   -  |pytorch| |tensorflow| `Proximal Policy Optimization (PPO) <rllib-algorithms.html#proximal-policy-optimization-ppo>`__
 
-   -  `Soft Actor Critic (SAC) <rllib-algorithms.html#soft-actor-critic-sac>`__
+   -  |tensorflow| `Soft Actor Critic (SAC) <rllib-algorithms.html#soft-actor-critic-sac>`__
 
 *  Derivative-free
 
-   -  `Augmented Random Search (ARS) <rllib-algorithms.html#augmented-random-search-ars>`__
+   -  |tensorflow| `Augmented Random Search (ARS) <rllib-algorithms.html#augmented-random-search-ars>`__
 
-   -  `Evolution Strategies <rllib-algorithms.html#evolution-strategies>`__
+   -  |tensorflow| `Evolution Strategies <rllib-algorithms.html#evolution-strategies>`__
 
 *  Multi-agent specific
 
-   -  `QMIX Monotonic Value Factorisation (QMIX, VDN, IQN) <rllib-algorithms.html#qmix-monotonic-value-factorisation-qmix-vdn-iqn>`__
-   -  `Multi-Agent Deep Deterministic Policy Gradient (contrib/MADDPG) <rllib-algorithms.html#multi-agent-deep-deterministic-policy-gradient-contrib-maddpg>`__
+   -  |pytorch| `QMIX Monotonic Value Factorisation (QMIX, VDN, IQN) <rllib-algorithms.html#qmix-monotonic-value-factorisation-qmix-vdn-iqn>`__
+   -  |tensorflow| `Multi-Agent Deep Deterministic Policy Gradient (contrib/MADDPG) <rllib-algorithms.html#multi-agent-deep-deterministic-policy-gradient-contrib-maddpg>`__
 
 *  Offline
 
-   -  `Advantage Re-Weighted Imitation Learning (MARWIL) <rllib-algorithms.html#advantage-re-weighted-imitation-learning-marwil>`__
+   -  |tensorflow| `Advantage Re-Weighted Imitation Learning (MARWIL) <rllib-algorithms.html#advantage-re-weighted-imitation-learning-marwil>`__
 
 Offline Datasets
 ----------------
@@ -180,3 +180,9 @@ TensorFlow 2.0
 ~~~~~~~~~~~~~~
 
 RLlib currently runs in ``tf.compat.v1`` mode. This means eager execution is disabled by default, and RLlib imports TF with ``import tensorflow.compat.v1 as tf; tf.disable_v2_behaviour()``. Eager execution can be enabled manually by calling ``tf.enable_eager_execution()`` or setting the ``"eager": True`` trainer config.
+
+.. |tensorflow| image:: tensorflow.png
+    :width: 16
+
+.. |pytorch| image:: pytorch.png
+    :width: 16
diff --git a/doc/source/rllib.rst b/doc/source/rllib.rst
index 8250117f2..4adee20e8 100644
--- a/doc/source/rllib.rst
+++ b/doc/source/rllib.rst
@@ -10,7 +10,7 @@ To get started, take a look over the `custom env example <https://github.com/ray
 RLlib in 60 seconds
 -------------------
 
-The following is a whirlwind overview of RLlib. For a more in-depth guide, see also the `full table of contents <rllib-toc.html>`__ and `RLlib blog posts <rllib-examples.html#blog-posts>`__. You may also want to skim the `list of built-in algorithms <rllib-toc.html#algorithms>`__.
+The following is a whirlwind overview of RLlib. For a more in-depth guide, see also the `full table of contents <rllib-toc.html>`__ and `RLlib blog posts <rllib-examples.html#blog-posts>`__. You may also want to skim the `list of built-in algorithms <rllib-toc.html#algorithms>`__. Look out for the |tensorflow| and |pytorch| icons to see which algorithms are available for each framework.
 
 Running RLlib
 ~~~~~~~~~~~~~
@@ -102,3 +102,9 @@ RLlib provides ways to customize almost all aspects of training, including the `
 .. image:: rllib-components.svg
 
 To learn more, proceed to the `table of contents <rllib-toc.html>`__.
+
+.. |tensorflow| image:: tensorflow.png
+    :width: 24
+
+.. |pytorch| image:: pytorch.png
+    :width: 24
diff --git a/doc/source/tensorflow.png b/doc/source/tensorflow.png
new file mode 100644
index 0000000000000000000000000000000000000000..05f9832e9f9b187d070c90f34ed7d004b7458e71
GIT binary patch
literal 2685
zcmai0YdF&lAO6pBXwGxUsxf6Y=Vc*LW=)ilQwk|!B9cuphcpb6^Bx+S;W3Avh#HYo
z5+;Wn6QLZEdK@w;hj_fb*L%I+-s^fl{O<eu{q8UK{o%f@U!t9@r6goO1ONcZGgbtL
z-%R`Wh>85hS0_EY0RTccLoj!uzF*7}4{~}7<KO)_X(NP2GwwS_PJxw_Q?J%Y`@gjx
z*YL715!P{RPtCOwnkF6kTo@CbdEp*`sdW7%xxCW$kry0ZQ>OD?<GeN;3Hbr4kjl2J
zhTxf#l{}PIioze|W4+6hzjm%!>8mybuSdREYg#s*56>2=zzG$cKe+z<qt3Md@tkx}
zdT)F_@v-iV5Po}FX4n0-%eJ$>(pA9w!L<4Jc_(bNBEH<R)1*xWj%X{c=2^2=lXbo5
zpmfRj%9#gFYbK_j!Qh4n5S}5kPspa8?U&mJfLqc!`cMg)&eMWx9#Gk9A2?Qw`pW0j
zaxXQ#FO<{Q7LLX7-mG9>&uSe799Je5m$Ki%ccBBN2zF{e*C0R*|2%Ug8I=AI`wz7_
z?rDt770X%d<qW#b?q<JC1dF$Kt$GBL2R20;@%VULiP8%;&p1_aDoiafa;av_1dPbT
zm7&rUE2d@dbncRhDnb2W5^u-D7^eTkV9?9UXZ{lGs?bwuP-_L0K)3P>oLD219v{vj
zUrMf0JhoNfVwExd0&TXUl)%3eF6uSPhnU3+_l1%sg-VWad`zRhCFgg+KFX!Z++NAT
z(tO-si@WxEfDsB_P3ggPDWzgODN_1@kX|v(xv7ickq<ye{dC^_%?u2e62aklQOE5K
zfqet_pRl(45XVI}kl@ksB;zg$P0xSBL1Zv(>`fD%bKt|R=UW8xFDo{OkHsifkmO`b
zx&l66r2BcWZ+@_S{b?!miwx$0(Dj)j@8Vh4qUF-Pbl*V}je+`}ZfSi8P%#D73`@c+
zL=OmT*&DUCDOVya?>v<!u|^y!9!Zh0nw@~J#;Y2p<qB0|nojk}JF%rA=1Ivp$GA|F
z>X!?;+-u#?pf?YpP?xYq0j4T_Tgy1#Py{=<Dph28`26E&1#LjS)wM&rYb=@%_HFcL
zY8!0tL*0w}v=Ti+elZdpfB^b8*?5>9)CU=~_;TcW|M1vO!gJ`f!3}Q9ZDXD+kk^(G
zj<ul%fZzP7(>)V;6UV6gW?67FNL9u6MDWuR>4G(-vr_G73#tT}d{S|?33#)$3_ca$
z@Et`=c|6@vZI6s=T(p*+f;PG&A4<@nml2c}PjX<*zp7-&HkI9CYfM8*!{g%oN}xZC
zIII3_DF2;(>d!kT@|(92KOKc{ERLMs??Aq*`VjbX%XSdz%;X>UsB_Wx+ll?>lU<8_
zz*_@$e$S5h88Ge|Hu_bVWSYzUsbYkQZ|KJa*%_DSzEfKPlvSh3cjPGD+P_Uh2Gdvr
z)TE)ANX;gEkKQ)JXsk4Jt;#Uyy6azn&Q-M?7aT*&&r;{jy644os+)0JnxKw&w;dVT
z6YS6vof!e>%<ReQ4*n!K-j;GR{+zU!eJKA(@`@UGtinGhTOYV3wR79hx|1ZXJbyb}
z6dSeW$@q$CHGS+KcIh42b`S_6H#44~^`L(a8)%ku`iNr31CQDeMhn)1EY7{}nv9^0
z1(c{%&XDV^aXl+hXXI6;peKh7+;38Tgxk?1PSgQISDWsx#!Bf;N`I|S@<q|T8}RPy
z-V!ugH-G_%BDB{41_r2u5wm}fITwuaXUF4LeHt~5fbjCN1yY4KDpo)psa--9rWkg}
z%_X3}{ZatFz5*qE{YJ<^)M737kFg{Jo$rOQ4NuFGK%BMfup&;msM4@y{H}D>Sw)AO
z^X5PE1!>0;q-o=~4D{-Q<1^(Z-+xIKMZB%OJKVeKwkfAIx4F?;s(oKxARqK}L|=#;
zp~@FA5M@|wv1B@4AG$C4aw8w)seaS3(+?-C95IwN&lRVE2^yL0Y~=Yy_%M`+&>V81
zQw*z+j=?nV`d@cJNyWM0k6KKkgJQA&T&$QUgql{|awTUWl@Xsm7zvbTdpF!^n7mLf
zZ7mT4-YC;+jbY8e(#BWBUU74=bIPo}59+~%>2D9dwwt<swC0+036xfWYt|;!JMqSQ
zauItReX}B$NoGZMoEB>vCL01ABDnOrC9{y_M;w1b=~r`+`H9t7STS<?8HKxmn?cr;
zCj}k&wegp>SG+0P1+Moo)~@9+rD@zZ!nmr!a?@P9y>F>LV;)(YyDyhP$r`T0UB7c@
zO?}P0j$;|dB<pIUe?>e%hc^*dLqLbUVP4+GFAUcrZNbM9x4EZQFclQI+-ABeI%yDu
z>uF*QbXmE)4B-Xub5v_P>)sygI$t04TG&%Jzq7_kX3l}0a^>?g9&3u9SP+2hTK6`7
z>#z_wmEBBjRBO)hf!G@^T<>xuTX8S(y;iZ!AXAaVxHt|N(b&ajs~IFSudFn>7^#X~
zJq-WlYQa!+@72`#w8D)IlKjIrWG7aLwTs$IgA>rj@k87UxX-|4TK|y@slz?QjzR7Z
zcF`H#_F{SyMKYy&73><;t}r(#4{=|R?C6$D{AOZw8-#C*#vJqn|Cp$>Bxe!7jwRa^
zxKdqFJWg=KGji?s9D!ppc$<D=aY$r2*nbbxO@~l#a<tW1_!>T{;1hy~WWD>LtNW#U
z2ro$^f(j2i@FiYwl6(gYE(HL4E`Vh@gAE%g9M*e-10uK*9}##$pnI2-6c&g{iHQUL
z0D0&GLVkId8i!BP27%67HN_!*WgrjL7H=S1;y?0#D^ucaHSra-KCa<uJ|YnXnSh!d
zcnOHHKc}OxWlY#XQTBqZ<q<$=zwEtHTK&?8+rB<D58%7O9_*;Dg}m&H3Hn>Hi5UhM
z6mFGlB&fbh)CY~qRXes4;idG4@&cou`b>09OM(ba#=I^=QG=Vk6DTTp+$<q3f>V#i
zB*7h8Fqv4pXU~PFFTXptqYpgAnvF^%&jBfP#|6n&!wDF~9CSl)<Ab8me+{dH|3`}d
zcU;u~>_eXO&u^W3;vbdP^e}8O!UIT;ljo&*62gq!-O66j-`i<=tCyZF8lUz8zC&W>
z8v8<Q3-b=qL$U9#Xh@0`akJZ_l-xcpN@hlkN)P*Ktn1Uv4ke$`|GcG8n1i+^8ES3I
zmqykaJKbNE4T}HJc|?K>ncg~jWl(wQ>b)_w4ZRD38wxVw;uNh-v_*^AAZC6%@oBts
zN2m7XpG5-RpAwmFWkosUmXc;_m-BY?%I@hp*UwD7;>GW48>IG4z+oZvl8K|WQU~Im
zY~b0w6RjrIl(>nu(+gnlLt@>d8d;gLNEND9qH>|aoURAa5SAaXwC5}`)Y=#|OE&Hw
z?5=H)BQbhrMRK}PV}{NylaTwHruQ(p=%h=*>QB^eimSfK4C~2+{`6T;BJIG;9XbLF
z@Q*gMlxxKJA#2R+CktG<gHS=sgai|!oTZ;ynlgZPlg^l0LbVD#J8}%&Azgk;zwp$y
it`Wma@bUL2xx2e$q;mImk@C6U3vkB5mhc$w5%Vv*s@yaH

literal 0
HcmV?d00001