* Qmix fix.
-Current version of double Q learning is incorrect; it selects actions
at timestep t instead of t+1 when computing the t+1 Q value.
* Allow extra obs dict keys
* Move Q-value-computing replay code to own function
* Run the autoformatter
* use better terms in comments ("policy" network instead of "live" network)