wassname/pyrobolearn
1
0
Fork 0
mirror of https://github.com/wassname/pyrobolearn.git synced 2026-09-09 11:31:38 +08:00
Code Issues Packages Projects Releases Wiki Activity

add readmes + update optimizers

Browse Source
This commit is contained in:
Brian Delhaisse
2019-03-18 08:18:46 +01:00
parent bd9d3b7868
commit 39e0d034ef
41 changed files with 1885 additions and 803 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pyrobolearn/__init__.py
+1 -1
View File
@@ -53,7 +53,7 @@ import tasks
# import metrics
# import optimizers
# import optim
# import optimizers
# import algos
import algos
pyrobolearn/actions/README.md
+8 -4
View File
@@ -2,9 +2,13 @@
The `Action` is produced by the policy in response to a certain state/observation. From a programming point of view, compared to the `State` class, the action is a setter object. Thus, they have a very close relationship and share many functionalities. Some actions are mutually exclusive and cannot be executed at the same time.
An action is defined as something that affects the environment; that forces the environment to go to the next state. For instance, an action could be the desired joint positions, but also an abstract action such as 'open a door' which would then open a door in the simulator and load the next part of the world.
An action is defined as something that affects the environment; that forces the environment to go to the next state. For instance, an action could be the desired joint positions, but also an abstract action such as 'open a door' which would then open a door in the simulator and load the next part of the world. This would depend on how the user implemented his/her action class.
In the framework, the `Action` class is decoupled from the policy and environment rendering it more modular [1]. Nevertheless, the `Action` class still acts as a bridge between the policy and environment. In addition to be the output of a policy/controller, it can also be the input to some value estimators, dynamic models, reward functions, and so on.
In the framework, the `Action` class is decoupled from the policy and environment rendering it more modular and [flexible](https://en.wikipedia.org/wiki/Composition_over_inheritance). Nevertheless, the `Action` class still acts as a bridge between the policy and environment. In addition to be the output of a policy/controller, it can also be the input to some value estimators, dynamic models, reward functions, and so on.
References:
[1] "Wikipedia: Composition over Inheritance", https://en.wikipedia.org/wiki/Composition_over_inheritance
You can for instance call `JointPositionAction(robot)`, and this will use position control to set the given joint positions. By giving the robot as input to the `JointPositionAction` class, it will automatically get the number of joints that the given robot possesses. This can then later be useful for instance when building a certain learning model for a policy. For example, assume we want to use a multilayer perceptron as the policy. The number of units on the last layer depends on the number of joints of the considered robot. Using the `MLPPolicy(outputs=actions)` it will automatically sets the correct number of output units depending on the considered robot.
#### What to check/look next?
Check first the `states` folder if not already done, then the `approximators`, `policies`, `rewards`, and `envs` folders.
pyrobolearn/algos/README.md
+12
View File
@@ -0,0 +1,12 @@
## Algos
This folder contains the various learning algorithms. Learning algorithms describe how to optimize the (hyper-)parameters of a particular learning model using a given loss function and optimizer.
Learning algorithms should not be confused with the models.
They can be divided into two main categories:
* supervised / unsupervised learning algorithms: This type of algorithm describes how to update the (hyper-)parameters of a learning model given some input (and possibly output) data, a loss function to evaluate its performance, and an optimizer.
* reinforcement learning algorithms:
* In the model-free paradigm: the algorithm basically performs 3 main steps:
1. Exploration: the algorithm describes how the policy should explore in the given environment and collect the various states, actions and rewards.
2. Evaluation: it evaluates the actions taken by the policy using a certain estimator
3. Update: this step is similar to supervised/unsupervised learning algorithms, where it updates the (hyper-)parameters of the various approximators (e.g. policies, value approximators, etc) using the given loss function and optimizer.
pyrobolearn/approximators/README.md
+11
View File
@@ -0,0 +1,11 @@
## Approximators
This folder provides the various approximators used in the PRL framework. Approximators are a layer on top of the learning models and can accept as inputs/outputs the `State` and `Action` classes defined in this framework in addition to normal tensors. In contrast, the learning models are completely independent of the various classes defined in this framework and can thus be used in other projects.
Approximators can be used to define:
- policies which map states to actions
- value function approximators which map states (and possibly actions) to a scalar.
- dynamic transition function approximators which map states and actions to the next state. This is notably useful in the model-based reinforcement learning paradigm.
- reward function approximator which map states/actions to a scalar. This is notably useful in the inverse reinforcement learning paradigm.
Approximators can be trained / optimized using a loss function and an optimizer.
pyrobolearn/backends/README.md
+8
View File
@@ -0,0 +1,8 @@
## Backends
The general idea is to provide different backends such that different tensor frameworks can be used. These include for instance numpy (with autograd), pytorch, and tensorflow.
These frameworks use different data structures and different signatures for the methods.
It would be nice to have learning models that are more or less independent of the tensor framework as done in Keras.
This is mostly an idea that I had in a later stage, and thus is not operational for the moment. It would require to refactor a bit the code, as currently our code is coupled to the pytorch and numpy frameworks.
pyrobolearn/control/README.md
+3
View File
@@ -0,0 +1,3 @@
## Control processes/algorithms
This folder will contain in the future control processes/algorithms.
pyrobolearn/controllers/README.md
+7
View File
@@ -0,0 +1,7 @@
## Controllers
Controllers are basically policies that do not possess any (hyper-)parameters to optimize. They are manually coded by the user.
Planning (TODO):
- [ ] add some finite state machines
- [ ] add some classical locomotion controllers (based on ZMP, optimization, etc)
pyrobolearn/dynamics/README.md
+5
View File
@@ -0,0 +1,5 @@
## Dynamic models
Dynamic models are models that given the current state (or a history of states), and the current action compute the next state. If dynamic models are provided or trained, we are often in a model-based reinforcement learning (also known as optimal control) paradigm.
Dynamic models will be soon added to the framework.
pyrobolearn/envs/README.md
+31
View File
@@ -0,0 +1,31 @@
## Environments
This folder provides the environment as it described in an imitation / reinforcement learning setting. That is, given an action performed by an agent (i.e. policy), the environment computes and returns the next state and possibly a reward. A policy can then interact with this environment, and be trained.
The environment defines the world, the rewards, and the states that are returned by it. To allow our framework to be flexible, the world, rewards, states, and terminal conditions are decoupled from the environment, and defined outside of this one and then provided as inputs to the `Env` class. See the `worlds`, `states`, and `rewards` folders. We thus favor [composition over inheritance](https://en.wikipedia.org/wiki/Composition_over_inheritance). This is a different approach compared to what is usually done with the [OpenAI gym](https://github.com/openai/gym) framework, where a world, state, rewards are defined inside the class that inherits from `gym.Env`.
```python
from itertools import count
import time
import pyrobolearn as prl
sim = prl.simulators.BulletSim()
world = prl.worlds.BasicWorld(sim)
robot = prl.robots.loadRobot('wam') # try another robot such as 'coman' or 'littledog'
state = prl.states.JointPositionState(robot) + prl.states.JointVelocityState(robot) # you can add other states
reward = prl.rewards.<Reward>(<state, action, etc>) # define the reward/cost
env = prl.envs.Env(world, state, reward, terminal_condition=None) # the state, world, reward, and terminal condition are defined outside the environment
for t in count():
next_obs, rew, done, info = env.step() # this will ask the state to read or compute the next value, and perform a step in the simulator
time.sleep(1./240)
if (t % 240) == 0:
print(reward) # print the reward
```
#### What to check/look next?
Check first the `policies` and `tasks` folders.
pyrobolearn/experiments/README.md
+16
View File
@@ -0,0 +1,16 @@
## Experiments
In this folder, we define the `Experiment` class which is the highest-level class of our framework. More specifically, it allows to organize which tasks to run, which metrics to use, and allows to easily compare different models, algos, methods, and so on.
The idea would be to define everything (world, robot, states, actions, rewards, environment, policy, algorithm, etc) inside the class that inherits from the `Experiment` class, and we should be able to run it using few commands.
```python
experiment = MyExperiment(<args>)
print(experiment.description())
experiment.train()
experiment.test()
experiment.plot()
```
This would I hope ease the reproduction of experiments performed in the literature.
Experiments will be added in a later stage.
pyrobolearn/filters/README.md
+8
View File
@@ -0,0 +1,8 @@
## Filters
This folder provides different filters / state estimators. These include for the moment:
- Histogram filter
- Kalman filter
- Extended Kalman filter
- Unscented Kalman filter
- Particle Filter
pyrobolearn/filters/description.txt
-1
View File
@@ -1 +0,0 @@
filters --> state estimators
pyrobolearn/metrics/README.md
+6
View File
@@ -0,0 +1,6 @@
## Metrics
Different learning tasks use different metrics. For instance, in transfer learning other metrics are used to evaluate the task than in reinforcement learning.
This folder will contain in the future the various metrics. They should be able to collect various information, evaluate how well the learning task is performed, and plot the results.
By providing the different metrics, the user can select which metric he/she wants to use to evaluate his/her learning task.
pyrobolearn/models/README.md
+23 -1
View File
@@ -1,3 +1,25 @@
## Learning models
In this folder, we provide the various learning models. These can be categorized into two categories: movement primitives and general function approximators.
In this folder, we provide the various learning models. These can be categorized into two categories: movement primitives and general function approximators.
These include:
- Central pattern generators (CPG; the version provided by )
- Dynamic movement primitives (DMP)
- Probabilistic movement primitives (ProMP)
- Kernelized movement primitives (KMP)
- Linear models
- PCA models
- Polynomial models
- Gaussian mixture models (GMM) with its regression counterpart (GMR)
- Gaussian processes (GP; it uses/wraps the [GPyTorch](https://github.com/cornellius-gp/gpytorch) library)
- Neural networks (currently, only MLP are provided)
TODO:
- [ ] finish to implement the models
- [ ] provide multiple tests/examples for each model
- [ ] implement other models such as HMMs
#### what to check/look next?
Check the `approximators`, `policies`, `values`, and `dynamics` folders.
pyrobolearn/optim/optimizer.py
-786
View File
@@ -1,786 +0,0 @@
#!/usr/bin/env python
"""Provide various optimizers.
Optimizers allows to optimize (i.e. minimize or maximize) a utility function (also known as objective function,
fitness function, loss, etc.) with or without constraints and bounds. Notably, it assumes the functions have some
parameters that the optimizer can update.
Mathematically, this is described as:
.. math:: \min_{x \in R^n} f(x)
subject to
.. math::
g_i(x) \geq 0, \quad i = 1,...,m
h_j(x) = 0, \quad j = 1,...,p
x_l \leq x \leq x_u
For trajectory optimization, check "An Introduction to Trajectory Optimization: How to do your own Direct Collocation".
"""
# TODO: trajectory optimization
from abc import ABCMeta, abstractmethod
# Numpy with autograd
import autograd.numpy as np # Thinly-wrapped numpy
from autograd import grad # The only autograd function you may ever need
# Scipy optimizer
import scipy
# Pytorch optimizers
import torch.nn as nn
import torch.optim as optim
# NLopt optimizers
try:
import nlopt
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install nlopt via `pip install nlopt`.")
# IPopt optimizer
try:
import ipopt
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install ipopt using the `pyrobolearn/scripts/install_ipopt.sh`."
"If ipopt is already installed, you can install the python wrapper via "
"`pip install ipopt`.")
# CVXOPT
# import cvxopt
# CVXPY: nice wrapper around cvxopt
# import cvxpy
# Quadprog
# import quadprog
# QPsolvers optimizers: unified Python interface for multiple QP solvers (cvxopt, cvxpy, quadprog,...)
try:
import qpsolvers
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install qpsolvers directly via 'pip install qpsolvers'.")
# Bayesian optimization
try:
import GPy
import GPyOpt
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install GPy/GPyOpt directly via 'pip install GPy' and "
"'pip install GPyOpt'.")
# CMA-ES
try:
import cma
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install CMA-ES or `pycma` directly via 'pip install cma'.")
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
##################################################################
# OPTIMIZER #
##################################################################
class Optimizer(object):
r"""Optimizer abstract class
This is an abstract class from which all optimizers inherit from. Most of the child optimizer classes are wrappers
around the original optimizer. This is to provide the same common interface to all the optimizers, and convert
seamlessly to the correct data types.
Optimizers are often given as a parameter to the learning algorithms, but can also be used of out of the box
directly on models. Several original optimizers can be given to the learning algorithm which will automatically
wrap the optimizer with the corresponding wrapper to provide a common interface.
In their most natural form, optimizers are used to ... optimization process which can be described mathematically
by:
.. math::
\min_{\theta} J(\theta) \mbox{ subj. to constraints}
\max_{\theta} J(\theta)
Several optimizers provides
The list of optimizers available are from the following libraries:
* nlopt
* ipopt
* torch.optim
* GPy / GPyOpt
* scipy.optimize
* qpsolvers (which includes cvxpy)
* cmaes
* pso
Each one of them expect a certain kind of type of the parameters.
Optimizers can be divided into:
* global vs local
* derivative-free vs gradient-based
* without constraints vs with (equality and/or inequality) constraints
Many implemented optimizers that can be found online are specific to a certain type of learning model.
If necessary, a conversion or wrapping process is carried out to make the optimizer work with the given learning
model.
"""
__metaclass__ = ABCMeta
def __init__(self, model, losses, hyperparameters):
"""
:param model: a certain type of model. If the original model is given instead of an instance of `Model`, then
the model will be wrapped appropriately.
:param losses:
:param hyperparameters:
"""
pass
##################################################################
# SCIPY #
##################################################################
class Scipy(object):
r"""Scipy optimizer
This uses the `scipy.optimize.minimize` to optimize a given objective function under various bounds and
constraints. Specifically, it consists of the minimization of a scalar function of one or more variables.
In general, the optimization problems are of the form:
.. math::
\min_{x \in R^n} f(x)
subject to
.. math::
g_i(x) \geq 0, \quad i = 1,...,m
h_j(x) = 0, \quad j = 1,...,p
where :math:`x` is a vector of one or more variables, :math:`g_i(x)` are the inequality constraints, and
:math:`h_j(x)` are the equality constrains.
Optionally, the lower and upper bounds for each element in :math:`x` can also be specified using the `bounds`
argument.
Several methods/optimizers are available:
-
Note that only 'COBYLA' and 'SLSQP' support constraints, where the former only supports inequality constraints.
References:
[1] scipy.optimize.minimize: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
"""
def __init__(self, method='SLSQP'):
"""
Initialize the scipy method
Args:
method (str, callable):
- 'Nelder-Mead' :ref:`(see here) <scipy.optimize.minimize-neldermead>`
- 'Powell' :ref:`(see here) <scipy.optimize.minimize-powell>`
- 'CG' :ref:`(see here) <scipy.optimize.minimize-cg>`
- 'BFGS' :ref:`(see here) <scipy.optimize.minimize-bfgs>`
- 'Newton-CG' :ref:`(see here) <scipy.optimize.minimize-newtoncg>`
- 'L-BFGS-B' :ref:`(see here) <scipy.optimize.minimize-lbfgsb>`
- 'TNC' :ref:`(see here) <scipy.optimize.minimize-tnc>`
- 'COBYLA' :ref:`(see here) <scipy.optimize.minimize-cobyla>`
- 'SLSQP' :ref:`(see here) <scipy.optimize.minimize-slsqp>`
- 'dogleg' :ref:`(see here) <scipy.optimize.minimize-dogleg>`
- 'trust-ncg' :ref:`(see here) <scipy.optimize.minimize-trustncg>`
- custom - a callable object (added in version 0.14.0),
"""
# define optimization method
# By default, it will be 'BFGS', 'L-BFGS-B', or 'SLSQP' depending on the constraints and bounds
# If constraints, it can only be 'COBYLA' or 'SLSQP'. COBYLA only supports inequality constraints.
self.method = method
def optimize(self, maxiter=1e6, verbose=True):
# define objective function to MINIMIZE
# f = lambda x: -(x.T.dot(C)).dot(x)
def f(x):
return -(x.T.dot(C)).dot(x)
# define initial guess
x0 = np.ones((M,)) # np.zeros((M,))
# define 1st constraints: norm of 1
constraints = [{'type': 'eq', 'fun': lambda x: x.T.dot(x) - 1, 'jac': None, 'args': ()}]
# define bounds: each vector u have a norm of 1 thus each parameter is between -1 and 1
bounds = [(-1., 1.)] * M
# optimize recursively
evals, evecs = [], []
messages = {}
options = {'maxiter': maxiter, 'disp': verbose}
for i in range(M):
if i != 0:
# add orthogonality constraint
constraints.append({'type': 'eq', 'fun': lambda u: u1.T.dot(u)})
# minimize --> it returns an instance of OptimizeResult
result = scipy.optimize.minimize(f, x0, args=(), method=self.method, jac=None, hess=None, bounds=bounds,
constraints=constraints, tol=None, callback=None, options=options)
print(result.success)
print(result.message)
print(result.fun)
print(result.x)
##################################################################
# Quadratic Programming #
##################################################################
# class CVXOPT(Optimizer):
# r"""Convex Optimizer
#
# Note: cvxpy module is a nice wrapper around cvxopt that follows paradigm of a disciplined convex programming.
#
# References:
# [1] Python Software for Convex Optimization: https://cvxopt.org/
# [2] Github repo: https://github.com/cvxopt/cvxopt
# """
# pass
#
#
# class CVXPY(Optimizer):
# r"""Convex Optimizer
#
# References:
# [1] CVXPY: http://www.cvxpy.org/
# [2] Github repo: https://github.com/cvxgrp/cvxpy
# """
# pass
#
#
# class QuadProg(object):
# r"""Quadprog
#
# References:
# [1] Github repo: https://github.com/rmcgibbo/quadprog
# """
# pass
class QP(object):
r"""Quadratic Programming solvers
This class uses the `qpsolvers` which is a unified Python interface for multiple QP solvers [1,2].
.. math::
\min_{x \in R^n} \frac{1}{2} x^T P x + q^T x
subject to
.. math::
Gx \leq h
Ax = b
where :math:`x` is the vector of optimization variables, the matrix :math:`P` and vector :math:`q` are used to
define any quadratic objective function on these variables, while the matrix-vector couples :math:`(G,h)` and
:math:`(A,b)` respectively define inequality and equality constraints. Vector inequalities apply coordinate by
coordinate [1].
- Dense solvers:
- CVXOPT
- CVXPY
- qpOASES
- quadprog
- Sparse solvers:
- ECOS as wrapped by CVXPY
- Gurobi
- MOSEK
- OSQP
Check the available solvers by calling `print(qpsolvers.available_solvers)`.
Notes: Many solvers (including CVXOPT, OSQP and quadprog) assume that `P` is a symmetric matrix, and may return
erroneous results when that is not the case. You can set ``sym_proj=True`` to project `P` on its symmetric part,
at the cost of some computation time.
References:
[1] QP in Python: https://scaron.info/blog/quadratic-programming-in-python.html
[2] Github repo: https://github.com/stephane-caron/qpsolvers
"""
def __init__(self, method='quadprog'):
"""
Initialize the QP solver.
Args:
method (str): ['cvxopt', 'cvxpy', 'ecos', 'gurobi', 'mosek', 'osqp', 'qpoases', 'quadprog']
"""
solvers = set(qpsolvers.available_solvers)
if len(solvers) == 0:
raise ValueError("No QP solvers have been found on this computer. Please install one of the QP modules")
if method not in solvers:
method = 'quadprog'
self.method = method
# check methods that require a symmetric matrix for P
methods = ['cvxopt', 'osqp', 'quadprog']
self.sym_proj = True if self.method in set(methods) else False
def is_symmetric(self, X, tol=1e-8):
return np.allclose(X, X.T, atol=tol)
def optimize(self, P, q, x0=None, G=None, h=None, A=None, b=None):
return qpsolvers.solve_qp(P, q, G, h, A, b, solver=self.method, initvals=x0, sym_proj=self.sym_proj)
##################################################################
# NLOPT #
##################################################################
class NLopt(object):
r"""Non-Linear Optimizer
Non-linear optimizers based on the `nlopt` libraries.
Here is a brief of lists of the current algorithms implemented:
*
Nonlinear optimization algos that can handle nonlinear inequality and EQUALITY constraints are:
- ISRES (Improved Stochastic Ranking Evolution Strategy) --> global derivative-free
- COBYLA (Constrained Optimization BY Linear Approximations) --> local derivative-free
- SLSQP (Sequential Least-SQuares Programming) --> local gradient-based
- AUGLAG (AUGmented LAGrangian) --> global/local derivative-free/gradient based (determined based on the
subsidiary algo)
More information about:
- algorithms: https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/
References:
[1] NLopt: https://nlopt.readthedocs.io/en/latest/
[2] NLopt with Python: with Python: https://nlopt.readthedocs.io/en/latest/NLopt_Python_Reference/
[3] Github repo: https://github.com/stevengj/nlopt
"""
# def __init__(self, model, losses, hyperparameters, seed):
# super(NLopt, self).__init__(model, losses, hyperparameters)
def __init__(self, method, submethod=None, seed=None):
# define useful variables
self.results = {1: 'success', 2: 'stop_val reached', 3: 'ftol reached', 4: 'xtol reached',
5: 'maxeval reached', 6: 'maxtime reached', -1: 'failure', -2: 'invalid args',
-3: 'out of memory', -4: 'roundoff limited', -5: 'forced stop'}
# define random seed
nlopt.srand(seed)
# define which solver to use
def get_opt(method):
if method == 'ISRES':
return nlopt.opt(nlopt.GN_ISRES, M)
elif method == 'COBYLA':
return nlopt.opt(nlopt.LN_COBYLA, M)
elif method == 'SLSQP':
return nlopt.opt(nlopt.LD_SLSQP, M)
elif method == 'AUGLAG':
return nlopt.opt(nlopt.AUGLAG, M)
else:
raise NotImplementedError("The given method has not been implemented")
if method is None:
method = 'SLSQP'
self.opt = get_opt(method)
# define subsolver to use (if we use the AUGLAG method)
if method == 'AUGLAG':
if submethod is None:
submethod = 'SLSQP'
elif submethod == 'AUGLAG':
raise ValueError("Submethod should be different from AUGLAG")
subopt = get_opt(submethod)
subopt.set_lower_bounds(-1)
subopt.set_upper_bounds(1)
# subopt.set_ftol_rel(1e-2)
# subopt.set_maxeval(100)
self.opt.set_local_optimizer(subopt)
def optimize(self):
# define objective function and its gradient
def f(x, grad):
if grad.size > 0:
grad[:] = 2 * x.T.dot(C)
return x.T.dot(C).dot(x)
# define objective function to maximize
self.opt.set_max_objective(f)
# if nlopt.GN_ISRES, we can define the population size
self.opt.set_population(0) # by default for ISRES: pop=20*(M+1)
# define bound constraints (should be between -1 and 1 because the norm should be 1)
self.opt.set_lower_bounds(-1.)
self.opt.set_upper_bounds(1.)
# define norm constraint and its gradient
def c1(x, grad):
if grad.size > 0:
grad[:] = 2 * x
return (x.T.dot(x) - 1)
# define orthogonal constraint
class OrthogonalConstraint(object):
def __init__(self, v):
self.v = np.copy(v)
def constraint(self, x, grad):
if grad.size > 0:
grad[:] = self.v
return (x.T.dot(self.v))
# define equality constraints
opt.add_equality_constraint(c1, 0)
# opt.add_equality_mconstraint(constraints, tol)
# define stopping criteria
# opt.set_stopval(stopval)
opt.set_ftol_rel(1e-8)
# opt.set_xtol_rel(1e-4)
opt.set_maxeval(100000) # nb of iteration
opt.set_maxtime(2) # time in secs
# define initial value
x0 = np.array([0.1] * M) # important that the initial value != 0 for the computation of the grad!
evals, evecs, msgs = [], [], {}
for i in range(M):
# add constraint
if i > 0:
c = OrthogonalConstraint(x)
opt.add_equality_constraint(c.constraint, 0)
# optimize
try:
x = opt.optimize(x0)
except nlopt.RoundoffLimited as e:
pass
# save values
evecs.append(x) # param vector
evals.append(opt.last_optimum_value()) # max value
msgs[i] = nlopt_results[opt.last_optimize_result()]
##################################################################
# IPOPT #
##################################################################
class NormConstraint(object):
def __init__(self):
pass
def constraint(self, x):
return x.T.dot(x)
def jacobian(self, x):
return 2 * x
class OrthogonalConstraint(object):
def __init__(self, v):
self.v = np.copy(v)
def constraint(self, x):
return x.T.dot(self.v)
def jacobian(self, x):
return self.v
class _IPopt(object):
def __init__(self, verbose=True):
self.verbose = verbose
self.iter_count = 0
self.constraints = []
def add_constraint(self, constraint):
self.constraints.append(constraint)
def objective(self, x):
# objective fct to minimize
return -x.T.dot(C).dot(x)
def gradient(self, x):
# grad of the objective fct
return -2 * x.T.dot(C)
def constraints(self, x):
return np.array([c.constraint(x) for c in self.constraints])
def jacobian(self, x):
return np.array([c.jacobian(x) for c in self.constraints])
# def hessian(self, x):
# pass
def intermediate(self, alg_mod, iter_count, obj_value, inf_pr, inf_du, mu, d_norm,
regularization_size, alpha_du, alpha_pr, ls_trials):
if self.verbose:
print("Objective value at iteration #%d: %g" % (iter_count, obj_value))
self.iter_count = iter_count
class IPopt(Optimizer):
r"""Interior-Point optimizer
This is a wrapper around the `ipopt` library. It can be used to solve general nonlinear programming problems of
the form:
.. math::
\min_{x \in R^n} f(x)
subject to
.. math::
g_L \leq g(x) \leq g_U
x_L \leq x \leq x_U
where :math:`x` are the optimization variables (possibly with upper an lower bounds, :math:`x_U` and :math:`x_L`
respectively), :math:`f(x)` is the objective function and :math:`g(x)` are the general nonlinear constraints.
The constraints, :math:`g(x)`, have lower and upper bounds. Note that equality constraints can be specified
by setting :math:`g^i_L = g^i_U`.
More info:
- Check the documentation of the `ipopt.problem` method
References:
[1] "On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear
programming", Wachter and Biegler, 2004
[2] Ipopt: https://projects.coin-or.org/Ipopt
[3] Ipopt in Python: https://pythonhosted.org/ipopt/
[4] Repos: https://github.com/coin-or/Ipopt and https://pypi.org/project/ipopt/
"""
def __init__(self, model, losses, hyperparameters):
super(IPopt, self).__init__(model, losses, hyperparameters)
def optimize(self):
# define initial value
x0 = np.array([0.1] * N) # important that the initial value != 0 for the computation of the grad!
# define (lower and upper) bound constraints
lb = [-1] * N
ub = [1] * N
# define constraints; if upper and lower constraints (resp. cu and cl) are equal then equality constraint
cl = [1] + [0] * (N - 1)
cu = [1] + [0] * (N - 1)
# create ipopt (which contains the objective function, its gradients, and constraints)
opt = _IPopt(verbose=False)
opt.add_constraint(NormConstraint())
opt.add_constraint(OrthogonalConstraint(x))
# define the nonlinear optimization problem
nlp = ipopt.problem(n=N, m=len(cl[:i]), problem_obj=opt, lb=lb, ub=ub, cl=cl[:i], cu=cu[:i])
# solve problem
x, info = nlp.solve(x0)
return x
##################################################################
# PYTORCH OPTIMIZERS #
##################################################################
class PyTorchOpt(Optimizer):
r"""PyTorch Optimizers
This is a wrapper around the optimizers from pytorch.
"""
def __init__(self, model, losses, hyperparameters):
super(PyTorchOpt, self).__init__(model, losses, hyperparameters)
def add_constraint(self):
# it will add a constraint as the augmented lagrangian
pass
class Adam(object):
r"""Adam Optimizer
References:
[1] "Adam: A Method for Stochastic Optimization", Kingma et al., 2014
"""
def __init__(self, learning_rate=1e-3, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False,
max_grad_norm=None): # 0.5
self.optimizer = None
self.learning_rate = learning_rate
self.betas = betas
self.eps = eps
self.weight_decay = weight_decay
self.amsgrad = amsgrad
self.max_grad_norm = max_grad_norm
def reset(self):
self.optimizer = None
def optimize(self, params, loss):
# create optimizer if necessary
if self.optimizer is None:
self.optimizer = optim.Adam(params, lr=self.learning_rate, betas=self.betas, eps=self.eps,
weight_decay=self.weight_decay, amsgrad=self.amsgrad)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class Adadelta(object):
r"""Adadelta Optimizer
References:
[1] "ADADELTA: An Adaptive Learning Rate Method", Zeiler, 2012
"""
def __init__(self, learning_rate=1., rho=0.9, eps=1e-6, weight_decay=0, max_grad_norm=None): #0.5
self.optimizer = None
self.learning_rate = learning_rate
self.rho = rho
self.eps = eps
self.weight_decay = weight_decay
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
if self.optimizer is None:
self.optimizer = optim.Adadelta(params, lr=self.learning_rate, rho=self.rho, eps=self.eps,
weight_decay=self.weight_decay)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class Adagrad(object):
r"""Adagrad Optimizer
References:
[1] "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization", Duchi et al., 2011
"""
def __init__(self, learning_rate=0.01, learning_rate_decay=0, weight_decay=0, initial_accumumaltor_value=0,
max_grad_norm=None): # 0.5
self.optimizer = None
self.learning_rate = learning_rate
self.learning_rate_decay = learning_rate_decay
self.weight_decay = weight_decay
self.initial_accumulator_value = initial_accumumaltor_value
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
if self.optimizer is None:
self.optimizer = optim.Adagrad(params, lr=self.learning_rate, lr_decay=self.learning_rate_decay,
weight_decay=self.weight_decay,
initial_accumulator_value=self.initial_accumulator_value)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class RMSprop(object):
r"""RMSprop
References:
[1] "RMSprop: Divide the gradient by a running average of its recent magnitude" (lecture 6.5), Tieleman and
Hinton, 2012
[2] "Generating Sequences With Recurrent Neural Networks", Graves, 2014
"""
def __init__(self, learning_rate=1e-2, alpha=0.99, eps=1e-8, weight_decay=0, momentum=0, centered=False,
max_grad_norm=None): # 0.5
self.optimizer = None
self.learning_rate = learning_rate
self.alpha = alpha
self.eps = eps
self.weight_decay = weight_decay
self.momentum = momentum
self.centered = centered
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
if self.optimizer is None:
self.optimizer = optim.RMSprop(params, lr=self.learning_rate, alpha=self.alpha, eps=self.eps,
weight_decay=self.weight_decay, momentum=self.momentum,
centered=self.centered)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class SGD(object):
r"""Stochastic Gradient Descent
References:
[1] "A Stochastic Approximation Method", Robbins and Monro, 1951
[2] "On the importance of initialization and momentum in deep learning", Sutskever et al., 2013
"""
def __init__(self, learning_rate=1e-3, momentum=0, dampening=0, weight_decay=0, nesterov=False,
max_grad_norm=None): #0.5
self.optimizer = None
self.learning_rate = learning_rate
self.momentum = momentum
self.dampening = dampening
self.weight_decay = weight_decay
self.nesterov = nesterov
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
# create optimizer if necessary
if self.optimizer is None:
self.optimizer = optim.SGD(params, lr=self.learning_rate, momentum=self.momentum, dampening=self.dampening,
weight_decay=self.weight_decay, nesterov=self.nesterov)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
pyrobolearn/optimizers/README.md
+18
View File
@@ -0,0 +1,18 @@
## Optimizers
This folder provides the various optimizers. It mostly wraps already existing optimizers but provides a common API.
Optimizers allows to maximize/minimize a function with respect to its parameters/inputs under possibly various (equality and/or inequality) constraints with possibly different bounds on the parameters.
Optimizers:
- [`scipy.optimize`](https://docs.scipy.org/doc/scipy/reference/optimize.html)
- [`torch.optim`](https://pytorch.org/docs/stable/optim.html)
- [`nlopt`](https://nlopt.readthedocs.io/en/latest/)
- [`ipopt`](https://projects.coin-or.org/Ipopt)
- [`qpsolvers`](https://github.com/stephane-caron/qpsolvers)
- [`cma`](https://github.com/CMA-ES/pycma)
- [`GPyOpt`](https://sheffieldml.github.io/GPyOpt/)
TODO:
- [ ] clean and finish to implement the various optimizers + structure the code in a better way
pyrobolearn/optim/__init__.py → pyrobolearn/optimizers/__init__.py
View File
pyrobolearn/optim/cio.py → pyrobolearn/optimizers/cio.py
+1 -1
View File
@@ -62,4 +62,4 @@ class CIO(object):
end_effector_quat = self.robot.getEndEffectorOrientations()
def optimize(self):
pass
pass
pyrobolearn/optimizers/cma_optimizer.py
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env python
"""Provide a wrapper around the Covariance Adaptation Matrix (CMA) which uses an evolution strategy to optimize
the parameters.
References:
[1] https://github.com/CMA-ES/pycma
"""
import numpy as np
# CMA-ES
try:
import cma
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install CMA-ES or `pycma` directly via 'pip install cma'.")
from optimizer import Optimizer
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
# TODO: check the `pyrobolearn/algos/cmaes` algo
class CMAES(object):
pass
pyrobolearn/optimizers/gpyopt_optimizer.py
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env python
"""Provide a wrapper around the GPyOpt optimizers for Bayesian Optimization (BO).
References:
[1] https://sheffieldml.github.io/GPyOpt/
"""
import numpy as np
# Bayesian optimization
try:
import GPy
import GPyOpt
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install GPy/GPyOpt directly via 'pip install GPy' and "
"'pip install GPyOpt'.")
from optimizer import Optimizer
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
# TODO: check the `pyrobolearn/algos/bo` algo
class BayesianOptimizer(object):
pass
pyrobolearn/optimizers/ipopt_optimizer.py
+148
View File
@@ -0,0 +1,148 @@
#!/usr/bin/env python
"""Provide a wrapper around the non-linear Interior Point optimizer (IPopt).
References:
[1] https://projects.coin-or.org/Ipopt
"""
import numpy as np
# IPopt optimizer
try:
import ipopt
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install ipopt using the `pyrobolearn/scripts/install_ipopt.sh`."
"If ipopt is already installed, you can install the python wrapper via "
"`pip install ipopt`.")
from optimizer import Optimizer
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
class NormConstraint(object):
def __init__(self):
pass
def constraint(self, x):
return x.T.dot(x)
def jacobian(self, x):
return 2 * x
class OrthogonalConstraint(object):
def __init__(self, v):
self.v = np.copy(v)
def constraint(self, x):
return x.T.dot(self.v)
def jacobian(self, x):
return self.v
class _IPopt(object):
def __init__(self, verbose=True):
self.verbose = verbose
self.iter_count = 0
self.constraints = []
def add_constraint(self, constraint):
self.constraints.append(constraint)
def objective(self, x):
# objective fct to minimize
return -x.T.dot(C).dot(x)
def gradient(self, x):
# grad of the objective fct
return -2 * x.T.dot(C)
def constraints(self, x):
return np.array([c.constraint(x) for c in self.constraints])
def jacobian(self, x):
return np.array([c.jacobian(x) for c in self.constraints])
# def hessian(self, x):
# pass
def intermediate(self, alg_mod, iter_count, obj_value, inf_pr, inf_du, mu, d_norm,
regularization_size, alpha_du, alpha_pr, ls_trials):
if self.verbose:
print("Objective value at iteration #%d: %g" % (iter_count, obj_value))
self.iter_count = iter_count
class IPopt(Optimizer):
r"""Interior-Point optimizer
This is a wrapper around the `ipopt` library. It can be used to solve general nonlinear programming problems of
the form:
.. math::
\min_{x \in R^n} f(x)
subject to
.. math::
g_L \leq g(x) \leq g_U
x_L \leq x \leq x_U
where :math:`x` are the optimization variables (possibly with upper an lower bounds, :math:`x_U` and :math:`x_L`
respectively), :math:`f(x)` is the objective function and :math:`g(x)` are the general nonlinear constraints.
The constraints, :math:`g(x)`, have lower and upper bounds. Note that equality constraints can be specified
by setting :math:`g^i_L = g^i_U`.
More info:
- Check the documentation of the `ipopt.problem` method
References:
[1] "On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear
programming", Wachter and Biegler, 2004
[2] Ipopt: https://projects.coin-or.org/Ipopt
[3] Ipopt in Python: https://pythonhosted.org/ipopt/
[4] Repos: https://github.com/coin-or/Ipopt and https://pypi.org/project/ipopt/
"""
def __init__(self, model, losses, hyperparameters):
super(IPopt, self).__init__(model, losses, hyperparameters)
def optimize(self):
# define initial value
x0 = np.array([0.1] * N) # important that the initial value != 0 for the computation of the grad!
# define (lower and upper) bound constraints
lb = [-1] * N
ub = [1] * N
# define constraints; if upper and lower constraints (resp. cu and cl) are equal then equality constraint
cl = [1] + [0] * (N - 1)
cu = [1] + [0] * (N - 1)
# create ipopt (which contains the objective function, its gradients, and constraints)
opt = _IPopt(verbose=False)
opt.add_constraint(NormConstraint())
opt.add_constraint(OrthogonalConstraint(x))
# define the nonlinear optimization problem
nlp = ipopt.problem(n=N, m=len(cl[:i]), problem_obj=opt, lb=lb, ub=ub, cl=cl[:i], cu=cu[:i])
# solve problem
x, info = nlp.solve(x0)
return x
pyrobolearn/optimizers/nlopt_optimizer.py
+162
View File
@@ -0,0 +1,162 @@
#!/usr/bin/env python
"""Provide a wrapper around the Non-Linear optimizers (NLopt).
References:
[1] https://nlopt.readthedocs.io/en/latest/
"""
import numpy as np
# NLopt optimizers
try:
import nlopt
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install nlopt via `pip install nlopt`.")
from optimizer import Optimizer
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
class NLopt(object):
r"""Non-Linear Optimizer
Non-linear optimizers based on the `nlopt` libraries.
Here is a brief of lists of the current algorithms implemented:
*
Nonlinear optimization algos that can handle nonlinear inequality and EQUALITY constraints are:
- ISRES (Improved Stochastic Ranking Evolution Strategy) --> global derivative-free
- COBYLA (Constrained Optimization BY Linear Approximations) --> local derivative-free
- SLSQP (Sequential Least-SQuares Programming) --> local gradient-based
- AUGLAG (AUGmented LAGrangian) --> global/local derivative-free/gradient based (determined based on the
subsidiary algo)
More information about:
- algorithms: https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/
References:
[1] NLopt: https://nlopt.readthedocs.io/en/latest/
[2] NLopt with Python: with Python: https://nlopt.readthedocs.io/en/latest/NLopt_Python_Reference/
[3] Github repo: https://github.com/stevengj/nlopt
"""
# def __init__(self, model, losses, hyperparameters, seed):
# super(NLopt, self).__init__(model, losses, hyperparameters)
def __init__(self, method, submethod=None, seed=None):
# define useful variables
self.results = {1: 'success', 2: 'stop_val reached', 3: 'ftol reached', 4: 'xtol reached',
5: 'maxeval reached', 6: 'maxtime reached', -1: 'failure', -2: 'invalid args',
-3: 'out of memory', -4: 'roundoff limited', -5: 'forced stop'}
# define random seed
nlopt.srand(seed)
# define which solver to use
def get_opt(method):
if method == 'ISRES':
return nlopt.opt(nlopt.GN_ISRES, M)
elif method == 'COBYLA':
return nlopt.opt(nlopt.LN_COBYLA, M)
elif method == 'SLSQP':
return nlopt.opt(nlopt.LD_SLSQP, M)
elif method == 'AUGLAG':
return nlopt.opt(nlopt.AUGLAG, M)
else:
raise NotImplementedError("The given method has not been implemented")
if method is None:
method = 'SLSQP'
self.opt = get_opt(method)
# define subsolver to use (if we use the AUGLAG method)
if method == 'AUGLAG':
if submethod is None:
submethod = 'SLSQP'
elif submethod == 'AUGLAG':
raise ValueError("Submethod should be different from AUGLAG")
subopt = get_opt(submethod)
subopt.set_lower_bounds(-1)
subopt.set_upper_bounds(1)
# subopt.set_ftol_rel(1e-2)
# subopt.set_maxeval(100)
self.opt.set_local_optimizer(subopt)
def optimize(self):
# define objective function and its gradient
def f(x, grad):
if grad.size > 0:
grad[:] = 2 * x.T.dot(C)
return x.T.dot(C).dot(x)
# define objective function to maximize
self.opt.set_max_objective(f)
# if nlopt.GN_ISRES, we can define the population size
self.opt.set_population(0) # by default for ISRES: pop=20*(M+1)
# define bound constraints (should be between -1 and 1 because the norm should be 1)
self.opt.set_lower_bounds(-1.)
self.opt.set_upper_bounds(1.)
# define norm constraint and its gradient
def c1(x, grad):
if grad.size > 0:
grad[:] = 2 * x
return (x.T.dot(x) - 1)
# define orthogonal constraint
class OrthogonalConstraint(object):
def __init__(self, v):
self.v = np.copy(v)
def constraint(self, x, grad):
if grad.size > 0:
grad[:] = self.v
return (x.T.dot(self.v))
# define equality constraints
self.opt.add_equality_constraint(c1, 0)
# opt.add_equality_mconstraint(constraints, tol)
# define stopping criteria
# self.opt.set_stopval(stopval)
self.opt.set_ftol_rel(1e-8)
# opt.set_xtol_rel(1e-4)
self.opt.set_maxeval(100000) # nb of iteration
self.opt.set_maxtime(2) # time in secs
# define initial value
x0 = np.array([0.1] * M) # important that the initial value != 0 for the computation of the grad!
evals, evecs, msgs = [], [], {}
for i in range(M):
# add constraint
if i > 0:
c = OrthogonalConstraint(x)
self.opt.add_equality_constraint(c.constraint, 0)
# optimize
try:
x = self.opt.optimize(x0)
except nlopt.RoundoffLimited as e:
pass
# save values
evecs.append(x) # param vector
evals.append(self.opt.last_optimum_value()) # max value
msgs[i] = nlopt_results[self.opt.last_optimize_result()]
pyrobolearn/optimizers/optimizer.py
+146
View File
@@ -0,0 +1,146 @@
#!/usr/bin/env python
"""Provide the abstract optimizer class.
Optimizers allows to optimize (i.e. minimize or maximize) a utility function (also known as objective function,
fitness function, loss, etc.) with or without constraints and bounds. Notably, it assumes the functions have some
parameters that the optimizer can update.
Mathematically, this is described as:
.. math:: \min_{x \in R^n} f(x)
subject to
.. math::
g_i(x) \geq 0, \quad i = 1,...,m
h_j(x) = 0, \quad j = 1,...,p
x_l \leq x \leq x_u
For trajectory optimization, check "An Introduction to Trajectory Optimization: How to do your own Direct Collocation".
"""
# TODO: trajectory optimization
from abc import ABCMeta, abstractmethod
# Numpy with autograd
# import autograd.numpy as np # Thinly-wrapped numpy
# from autograd import grad # The only autograd function you may ever need
import numpy as np
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
class Optimizer(object):
r"""Optimizer abstract class
This is an abstract class from which all optimizers inherit from. Most of the child optimizer classes are wrappers
around the original optimizer. This is to provide the same common interface to all the optimizers, and convert
seamlessly to the correct data types.
Optimizers are often given as a parameter to the learning algorithms, but can also be used of out of the box
directly on models. Several original optimizers can be given to the learning algorithm which will automatically
wrap the optimizer with the corresponding wrapper to provide a common interface.
In their most natural form, optimizers are used to ... optimization process which can be described mathematically
by:
.. math::
\min_{\theta} J(\theta) \mbox{ subj. to constraints}
\max_{\theta} J(\theta)
Several optimizers provides
The list of optimizers available are from the following libraries:
* nlopt
* ipopt
* torch.optim
* GPy / GPyOpt
* scipy.optimize
* qpsolvers (which includes cvxpy)
* cmaes
* pso
Each one of them expect a certain kind of type of the parameters.
Optimizers can be divided into:
* global vs local
* derivative-free vs gradient-based
* without constraints vs with (equality and/or inequality) constraints
Many implemented optimizers that can be found online are specific to a certain type of learning model.
If necessary, a conversion or wrapping process is carried out to make the optimizer work with the given learning
model.
"""
__metaclass__ = ABCMeta
def __init__(self, *args, **kwargs): # model, losses, hyperparameters):
"""
Initialize the optimizer.
"""
self.best_parameters = None
self.best_result = None
self.is_maximizing = True
##############
# Properties #
##############
@property
def best_parameters(self):
return self._best_parameters
@best_parameters.setter
def best_parameters(self, params):
self._best_parameters = params
@property
def best_result(self):
return self._best_result
@best_result.setter
def best_result(self, result):
self._best_result = result
@property
def is_maximizing(self):
return self._is_maximizing
@is_maximizing.setter
def is_maximizing(self, boolean):
self._is_maximizing = bool(boolean)
@property
def is_minimizing(self):
return not self.is_maximizing
###########
# Methods #
###########
def optimize(self, *args, **kwargs):
pass
#############
# Operators #
#############
def __repr__(self):
return self.__class__.__name__
def __str__(self):
return self.__class__.__name__
def __call__(self, *args, **kwargs):
return self.optimize(*args, **kwargs)
pyrobolearn/optimizers/qpsolvers_optimizer.py
+130
View File
@@ -0,0 +1,130 @@
#!/usr/bin/env python
"""Provide a wrapper around quadratic programming solvers.
References:
[1] https://github.com/stephane-caron/qpsolvers
"""
import numpy as np
# CVXOPT
# import cvxopt
# CVXPY: nice wrapper around cvxopt
# import cvxpy
# Quadprog
# import quadprog
# QPsolvers optimizers: unified Python interface for multiple QP solvers (cvxopt, cvxpy, quadprog,...)
try:
import qpsolvers
except ImportError as e:
raise ImportError(e.__str__() + "\n HINT: you can install qpsolvers directly via 'pip install qpsolvers'.")
from optimizer import Optimizer
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
# class CVXOPT(Optimizer):
# r"""Convex Optimizer
#
# Note: cvxpy module is a nice wrapper around cvxopt that follows paradigm of a disciplined convex programming.
#
# References:
# [1] Python Software for Convex Optimization: https://cvxopt.org/
# [2] Github repo: https://github.com/cvxopt/cvxopt
# """
# pass
#
#
# class CVXPY(Optimizer):
# r"""Convex Optimizer
#
# References:
# [1] CVXPY: http://www.cvxpy.org/
# [2] Github repo: https://github.com/cvxgrp/cvxpy
# """
# pass
#
#
# class QuadProg(object):
# r"""Quadprog
#
# References:
# [1] Github repo: https://github.com/rmcgibbo/quadprog
# """
# pass
class QP(object):
r"""Quadratic Programming solvers
This class uses the `qpsolvers` which is a unified Python interface for multiple QP solvers [1,2].
.. math::
\min_{x \in R^n} \frac{1}{2} x^T P x + q^T x
subject to
.. math::
Gx \leq h
Ax = b
where :math:`x` is the vector of optimization variables, the matrix :math:`P` and vector :math:`q` are used to
define any quadratic objective function on these variables, while the matrix-vector couples :math:`(G,h)` and
:math:`(A,b)` respectively define inequality and equality constraints. Vector inequalities apply coordinate by
coordinate [1].
- Dense solvers:
- CVXOPT
- CVXPY
- qpOASES
- quadprog
- Sparse solvers:
- ECOS as wrapped by CVXPY
- Gurobi
- MOSEK
- OSQP
Check the available solvers by calling `print(qpsolvers.available_solvers)`.
Notes: Many solvers (including CVXOPT, OSQP and quadprog) assume that `P` is a symmetric matrix, and may return
erroneous results when that is not the case. You can set ``sym_proj=True`` to project `P` on its symmetric part,
at the cost of some computation time.
References:
[1] QP in Python: https://scaron.info/blog/quadratic-programming-in-python.html
[2] Github repo: https://github.com/stephane-caron/qpsolvers
"""
def __init__(self, method='quadprog'):
"""
Initialize the QP solver.
Args:
method (str): ['cvxopt', 'cvxpy', 'ecos', 'gurobi', 'mosek', 'osqp', 'qpoases', 'quadprog']
"""
solvers = set(qpsolvers.available_solvers)
if len(solvers) == 0:
raise ValueError("No QP solvers have been found on this computer. Please install one of the QP modules")
if method not in solvers:
method = 'quadprog'
self.method = method
# check methods that require a symmetric matrix for P
methods = ['cvxopt', 'osqp', 'quadprog']
self.sym_proj = True if self.method in set(methods) else False
def is_symmetric(self, X, tol=1e-8):
return np.allclose(X, X.T, atol=tol)
def optimize(self, P, q, x0=None, G=None, h=None, A=None, b=None):
return qpsolvers.solve_qp(P, q, G, h, A, b, solver=self.method, initvals=x0, sym_proj=self.sym_proj)
pyrobolearn/optimizers/scipy_optimizer.py
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env python
"""Provide a wrapper around the Scipy optimizers.
References:
[1] https://docs.scipy.org/doc/scipy/reference/optimize.html
"""
import numpy as np
import scipy
from optimizer import Optimizer
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
class Scipy(object):
r"""Scipy optimizer
This uses the `scipy.optimize.minimize` to optimize a given objective function under various bounds and
constraints. Specifically, it consists of the minimization of a scalar function of one or more variables.
In general, the optimization problems are of the form:
.. math::
\min_{x \in R^n} f(x)
subject to
.. math::
g_i(x) \geq 0, \quad i = 1,...,m
h_j(x) = 0, \quad j = 1,...,p
where :math:`x` is a vector of one or more variables, :math:`g_i(x)` are the inequality constraints, and
:math:`h_j(x)` are the equality constrains.
Optionally, the lower and upper bounds for each element in :math:`x` can also be specified using the `bounds`
argument.
Several methods/optimizers are available:
-
Note that only 'COBYLA' and 'SLSQP' support constraints, where the former only supports inequality constraints.
References:
[1] scipy.optimize.minimize: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
"""
def __init__(self, method='SLSQP'):
"""
Initialize the scipy method
Args:
method (str, callable):
- 'Nelder-Mead' :ref:`(see here) <scipy.optimize.minimize-neldermead>`
- 'Powell' :ref:`(see here) <scipy.optimize.minimize-powell>`
- 'CG' :ref:`(see here) <scipy.optimize.minimize-cg>`
- 'BFGS' :ref:`(see here) <scipy.optimize.minimize-bfgs>`
- 'Newton-CG' :ref:`(see here) <scipy.optimize.minimize-newtoncg>`
- 'L-BFGS-B' :ref:`(see here) <scipy.optimize.minimize-lbfgsb>`
- 'TNC' :ref:`(see here) <scipy.optimize.minimize-tnc>`
- 'COBYLA' :ref:`(see here) <scipy.optimize.minimize-cobyla>`
- 'SLSQP' :ref:`(see here) <scipy.optimize.minimize-slsqp>`
- 'dogleg' :ref:`(see here) <scipy.optimize.minimize-dogleg>`
- 'trust-ncg' :ref:`(see here) <scipy.optimize.minimize-trustncg>`
- custom - a callable object (added in version 0.14.0),
"""
# define optimization method
# By default, it will be 'BFGS', 'L-BFGS-B', or 'SLSQP' depending on the constraints and bounds
# If constraints, it can only be 'COBYLA' or 'SLSQP'. COBYLA only supports inequality constraints.
self.method = method
def optimize(self, maxiter=1e6, verbose=True):
# define objective function to MINIMIZE
# f = lambda x: -(x.T.dot(C)).dot(x)
def f(x):
return -(x.T.dot(C)).dot(x)
# define initial guess
x0 = np.ones((M,)) # np.zeros((M,))
# define 1st constraints: norm of 1
constraints = [{'type': 'eq', 'fun': lambda x: x.T.dot(x) - 1, 'jac': None, 'args': ()}]
# define bounds: each vector u have a norm of 1 thus each parameter is between -1 and 1
bounds = [(-1., 1.)] * M
# optimize recursively
evals, evecs = [], []
messages = {}
options = {'maxiter': maxiter, 'disp': verbose}
for i in range(M):
if i != 0:
# add orthogonality constraint
constraints.append({'type': 'eq', 'fun': lambda u: u1.T.dot(u)})
# minimize --> it returns an instance of OptimizeResult
result = scipy.optimize.minimize(f, x0, args=(), method=self.method, jac=None, hess=None, bounds=bounds,
constraints=constraints, tol=None, callback=None, options=options)
print(result.success)
print(result.message)
print(result.fun)
print(result.x)
pyrobolearn/optimizers/torch_optimizer.py
+197
View File
@@ -0,0 +1,197 @@
#!/usr/bin/env python
"""Provide wrappers around the torch optimizers.
Compared to the optimizers present in the PyTorch library [1], which require to pass the parameters of the model when
instantiating them, here we can pass the parameters at a later stage.
References:
[1] https://pytorch.org/docs/stable/optim.html
"""
# Pytorch optimizers
import torch.nn as nn
import torch.optim as optim
from optimizer import Optimizer
__author__ = "Brian Delhaisse"
__copyright__ = "Copyright 2018, PyRoboLearn"
__credits__ = ["Brian Delhaisse"]
__license__ = "MIT"
__version__ = "1.0.0"
__maintainer__ = "Brian Delhaisse"
__email__ = "briandelhaisse@gmail.com"
__status__ = "Development"
# class PyTorchOpt(Optimizer):
# r"""PyTorch Optimizers
#
# This is a wrapper around the optimizers from pytorch.
# """
#
# def __init__(self, model, losses, hyperparameters):
# super(PyTorchOpt, self).__init__(model, losses, hyperparameters)
#
# def add_constraint(self):
# # it will add a constraint as the augmented lagrangian
# pass
class Adam(object):
r"""Adam Optimizer
References:
[1] "Adam: A Method for Stochastic Optimization", Kingma et al., 2014
"""
def __init__(self, learning_rate=1e-3, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False,
max_grad_norm=None): # 0.5
self.optimizer = None
self.learning_rate = learning_rate
self.betas = betas
self.eps = eps
self.weight_decay = weight_decay
self.amsgrad = amsgrad
self.max_grad_norm = max_grad_norm
def reset(self):
self.optimizer = None
def optimize(self, params, loss):
# create optimizer if necessary
if self.optimizer is None:
self.optimizer = optim.Adam(params, lr=self.learning_rate, betas=self.betas, eps=self.eps,
weight_decay=self.weight_decay, amsgrad=self.amsgrad)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class Adadelta(object):
r"""Adadelta Optimizer
References:
[1] "ADADELTA: An Adaptive Learning Rate Method", Zeiler, 2012
"""
def __init__(self, learning_rate=1., rho=0.9, eps=1e-6, weight_decay=0, max_grad_norm=None): #0.5
self.optimizer = None
self.learning_rate = learning_rate
self.rho = rho
self.eps = eps
self.weight_decay = weight_decay
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
if self.optimizer is None:
self.optimizer = optim.Adadelta(params, lr=self.learning_rate, rho=self.rho, eps=self.eps,
weight_decay=self.weight_decay)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class Adagrad(object):
r"""Adagrad Optimizer
References:
[1] "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization", Duchi et al., 2011
"""
def __init__(self, learning_rate=0.01, learning_rate_decay=0, weight_decay=0, initial_accumumaltor_value=0,
max_grad_norm=None): # 0.5
self.optimizer = None
self.learning_rate = learning_rate
self.learning_rate_decay = learning_rate_decay
self.weight_decay = weight_decay
self.initial_accumulator_value = initial_accumumaltor_value
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
if self.optimizer is None:
self.optimizer = optim.Adagrad(params, lr=self.learning_rate, lr_decay=self.learning_rate_decay,
weight_decay=self.weight_decay,
initial_accumulator_value=self.initial_accumulator_value)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class RMSprop(object):
r"""RMSprop
References:
[1] "RMSprop: Divide the gradient by a running average of its recent magnitude" (lecture 6.5), Tieleman and
Hinton, 2012
[2] "Generating Sequences With Recurrent Neural Networks", Graves, 2014
"""
def __init__(self, learning_rate=1e-2, alpha=0.99, eps=1e-8, weight_decay=0, momentum=0, centered=False,
max_grad_norm=None): # 0.5
self.optimizer = None
self.learning_rate = learning_rate
self.alpha = alpha
self.eps = eps
self.weight_decay = weight_decay
self.momentum = momentum
self.centered = centered
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
if self.optimizer is None:
self.optimizer = optim.RMSprop(params, lr=self.learning_rate, alpha=self.alpha, eps=self.eps,
weight_decay=self.weight_decay, momentum=self.momentum,
centered=self.centered)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
class SGD(object):
r"""Stochastic Gradient Descent
References:
[1] "A Stochastic Approximation Method", Robbins and Monro, 1951
[2] "On the importance of initialization and momentum in deep learning", Sutskever et al., 2013
"""
def __init__(self, learning_rate=1e-3, momentum=0, dampening=0, weight_decay=0, nesterov=False,
max_grad_norm=None): #0.5
self.optimizer = None
self.learning_rate = learning_rate
self.momentum = momentum
self.dampening = dampening
self.weight_decay = weight_decay
self.nesterov = nesterov
self.max_grad_norm = max_grad_norm
def optimize(self, params, loss):
# create optimizer if necessary
if self.optimizer is None:
self.optimizer = optim.SGD(params, lr=self.learning_rate, momentum=self.momentum, dampening=self.dampening,
weight_decay=self.weight_decay, nesterov=self.nesterov)
# optimize
self.optimizer.zero_grad()
loss.backward()
if self.max_grad_norm is not None:
nn.utils.clip_grad_norm_(params, self.max_grad_norm)
self.optimizer.step()
pyrobolearn/policies/README.md
+5
View File
@@ -0,0 +1,5 @@
## Policies
Policies in this framework are controllers that can be trained and map states to actions. They use directly the learning model or the `Approximator` class (which uses the learning model).
In this framework, `State` and `Action` instances should be given to the `Policy` which would infer its input and output dimensions and build the model with the correct number of inputs/outputs. In contrast to the learning model, the policy should know how to feed the various input states to the inner learning model, such that if a picture and joint states are given to the policy it knows where to feed the corresponding input observations.
pyrobolearn/processors/README.md
+7
View File
@@ -0,0 +1,7 @@
## Processors
Processors are functions that are applied to the inputs (respectively outputs) of an approximator/learning model before (respectively after) being processed by it. Processors might have parameters but these are not trainable/optimizable and are thus fixed and given at the beginning.
These processors allow for instance to scale the input and/or output of a learning model. They accept as inputs a State/Action, numpy array, or torch Tensor.
They can for instance be used to normalize the input state before it is fed to the policy.
pyrobolearn/recorders/README.md
+3
View File
@@ -0,0 +1,3 @@
## Recorders
Recorders record the states and actions.
pyrobolearn/rewards/README.md
+6 -1
View File
@@ -1,3 +1,8 @@
## Rewards
In this folder, we define the most common rewards used in reinforcement learning and optimization.
In this folder, we define the most common rewards/costs used in reinforcement learning.
#### what to check/look next?
Check the `envs` folder, which accepts the rewards.
pyrobolearn/robots/README.md
+95
View File
@@ -0,0 +1,95 @@
## Robots
This folder contains the various robots that can be used in the PRL framework. Currently, they can be loaded in the [PyBullet](https://pybullet.org/wordpress/) simulator. The corresponding URDFs have been downloaded from various repositories and updated/corrected if necessary (adding inertial tags, correcting inertia values, updating collision meshes, etc). All the robots inherit from the main `Robot` class, and can be loaded in the framework using:
```python
import pyrobolearn as prl
sim = prl.simulators.BulletSim()
robot = prl.robots.<RobotClass>(sim)
```
or
```python
import pyrobolearn as prl
sim = prl.simulators.BulletSim()
world = prl.worlds.BasicWorld(sim)
robot = world.loadRobot(<Robot_name_or_robot_class>)
```
The folder contains different kind of robots including manipulators, legged robots, wheeled robots, and others. It includes:
- [Aibo](https://github.com/dkotfis/aibo_ros)
- [Allegrohand](https://github.com/simlabrobotics/allegro_hand_ros)
- [Ant](https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/mjcf)
- Atlas: [1](https://github.com/openai/roboschool), [2](https://github.com/erwincoumans/pybullet_robots)
- [Ballbot](https://github.com/CesMak/bb)
- [Baxter](https://github.com/RethinkRobotics/baxter_common)
- BB8: [1](http://www.theconstructsim.com/bb-8-gazebo-model/), [2](https://github.com/eborghi10/BB-8-ROS)
- [Blackbird](https://hackaday.io/project/160882-blackbird-bipedal-robot)
- [Cartpole](https://github.com/bulletphysics/bullet3/blob/master/data/cartpole.urdf) but modified to be able to have multiple links specified at runtime
- Cassie: [1](https://github.com/UMich-BipedLab/Cassie_Model), [2](https://github.com/agilityrobotics/cassie-gazebo-sim), [3](https://github.com/erwincoumans/pybullet_robots)
- [Centauro](https://github.com/ADVRHumanoids/centauro-simulator)
- [Cogimon](https://github.com/ADVRHumanoids/iit-cogimon-ros-pkg)
- [Coman](https://github.com/ADVRHumanoids/iit-coman-ros-pkg)
- [Crab](https://github.com/tuuzdu/crab_project)
- [Cubli](https://github.com/xinsongyan/cubli)
- [Darwin](https://github.com/HumaRobotics/darwin_description)
- [e.Do](https://github.com/Comau/eDO_description)
- [epuck](https://github.com/gctronic/epuck_driver_cpp)
- [F10 racecar](https://github.com/erwincoumans/pybullet_robots/tree/master/data/f10_racecar)
- [Fetch](https://github.com/fetchrobotics/fetch_ros)
- [Franka Emika](https://github.com/frankaemika/franka_ros)
- [Half Cheetah](https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/mjcf)
- [Hopper](https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/mjcf)
- [Hubo](https://github.com/robEllenberg/hubo-urdf)
- [Humanoid](https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/mjcf)
- [Husky](https://github.com/husky/husky)
- [HyQ](https://github.com/iit-DLSLab/hyq-description)
- [HyQ2Max](https://github.com/iit-DLSLab/hyq2max-description)
- ICub: [1](https://github.com/robotology-playground/icub-models), [2](https://github.com/robotology-playground/icub-model-generator). There are currently few problems with this robot.
- [Jaco](https://github.com/JenniferBuehler/jaco-arm-pkgs)
- KR5: [1](https://github.com/a-price/KR5sixxR650WP_description), [2](https://github.com/ros-industrial/kuka_experimental)
- Kuka IIWA: [1](https://github.com/IFL-CAMP/iiwa_stack), [2](https://github.com/bulletphysics/bullet3/tree/master/data/kuka_iiwa)
- Kuka LWR: [1](https://github.com/CentroEPiaggio/kuka-lwr), [2](https://github.com/bulletphysics/bullet3/tree/master/data/kuka_lwr)
- [Laikago](https://github.com/erwincoumans/pybullet_robots)
- [Little Dog](https://github.com/RobotLocomotion/LittleDog)
- [Manipulator2D](https://github.com/domingoesteban/robolearn_robots_ros)
- [Minitaur](https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/quadruped)
- [Lincoln MKZ car](https://bitbucket.org/DataspeedInc/dbw_mkz_ros)
- [Morphex](https://gist.github.com/lanius/cb8b5e0ede9ff3b2b2c1bc68b95066fb)
- Nao: [1](https://github.com/ros-naoqi/nao_robot), and [2](https://github.com/ros-naoqi/nao_meshes)
- OpenDog: [1](https://github.com/XRobots/openDog), and [2](https://github.com/wiccopruebas/opendog_project)
- [Pepper](https://github.com/ros-naoqi/pepper_robot)
- [Phantom X](https://github.com/HumaRobotics/phantomx_description)
- [Pleurobot](https://github.com/KM-RoBoTa/pleurobot_ros_pkg)
- [PR2](https://github.com/pr2/pr2_common)
- [Quadcopter](https://github.com/wilselby/ROS_quadrotor_simulator)
- [Rhex](https://github.com/grafoteka/rhex)
- [RRbot](https://github.com/ros-simulation/gazebo_ros_demos)
- Sawyer: [1](https://github.com/RethinkRobotics/sawyer_robot), [2](https://github.com/erwincoumans/pybullet_robots)
- [SEA hexapod](https://github.com/alexansari101/snake_ws)
- [SEA snake]( https://github.com/alexansari101/snake_ws)
- [Soft hand](https://github.com/CentroEPiaggio/pisa-iit-soft-hand)
- [Swimmer](https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/mjcf)
- [Valkyrie](https://github.com/openhumanoids/val_description)
- [Walker 2D](https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/mjcf)
- [Walk-man](https://github.com/ADVRHumanoids/iit-walkman-ros-pkg)
- [Wam](https://github.com/jhu-lcsr/barrett_model)
- [Youbot](https://github.com/youbot): this includes the youbot base without any arms, one kuka arm, 2 kuka arms, and the kuka arm without the wheeled base.
Here is a list of robots that I plan to add at one point (some of them require to simulate some fluid dynamics, as done in the `quadcopter` class) but can interest already some people:
- [ ] [ANYmal](https://www.anymal-research.org/): I am currently not sure if I can release the URDF of this robot, and thus I will not do it until it is officially released by ETH.
- [ ] [ROS robots](https://robots.ros.org/): I plan to provide soon the robots listed on this website (I am currently cleaning the URDFs of some of them)
- [ ] [rotors-simulator](https://github.com/ethz-asl/rotors_simulator)
- [ ] [uuv-simulator](https://github.com/uuvsimulator/uuv_simulator)
- [ ] [usv-simulator](https://github.com/OUXT-Polaris/ros_ship_packages)
Note that currently, we load directly the URDF from the xml/urdf file using the simulator, but later we might first parse it beforehand and allow the user to add, remove, or change few links at runtime.
TODO:
- [ ] correct still few URDFs (some of them have inertia values that are too high!)
- [ ] finish to implement few methods
- [ ] move all the urdfs outside the pyrobolearn framework: put it in another repo or in a dropbox/google drive.
pyrobolearn/robots/__init__.py
+1 -1
View File
@@ -31,7 +31,7 @@ from humanoid import Humanoid
from aibo import Aibo
from minitaur import Minitaur
from littledog import LittleDog
# from anymal import ANYmal
# from anymal import ANYmal
from hyq import HyQ
from hyq2max import HyQ2Max
from opendog import OpenDog
pyrobolearn/robots/manipulator2d.py
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env python
"""Provide 2d manipulators.
"""
import os
import numpy as np
from manipulator import ManipulatorRobot
class Manipulator2D(ManipulatorRobot):
r"""2D manipulator robot
References:
[1] https://github.com/domingoesteban/robolearn_robots_ros
"""
def __init__(self,
simulator,
init_pos=(0, 0, 0),
init_orient=(0, 0, 0, 1),
useFixedBase=False,
scaling=1.,
urdf_path=os.path.dirname(__file__) + '/urdfs/manipulator2d/manipulator2d.urdf'):
# check parameters
if init_pos is None:
init_pos = (0., 0., 0.)
if len(init_pos) == 2: # assume x, y are given
init_pos = tuple(init_pos) + (0.,)
if init_orient is None:
init_orient = (0, 0, 0, 1)
if useFixedBase is None:
useFixedBase = False
super(Manipulator2D, self).__init__(simulator, urdf_path, init_pos, init_orient, useFixedBase, scaling)
self.name = 'manipulator2d'
# Test
if __name__ == "__main__":
from itertools import count
from pyrobolearn.simulators import BulletSim
from pyrobolearn.worlds import BasicWorld
# Create simulator
sim = BulletSim()
# create world
world = BasicWorld(sim)
# create robot
robot = Manipulator2D(sim, init_pos=(0, -0.25, 0))
robot1 = Manipulator2D(sim, init_pos=(0, 0.25, 0))
robot.printRobotInfo()
# Position control using sliders
# robot.addJointSlider()
# run simulator
for _ in count():
# robot.updateJointSlider()
world.step(sleep_dt=1./240)
pyrobolearn/robots/urdfs/manipulator2d/LICENSE
+27
View File
@@ -0,0 +1,27 @@
Copyright (c) 2019, Domingo Esteban (domingo.esteban@iit.it)
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. All advertising materials mentioning features or use of this software
must display the following acknowledgement:
This product includes software developed by the <organization>.
4. Neither the name of the <organization> nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ''AS IS'' AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
pyrobolearn/robots/urdfs/manipulator2d/manipulator2d.urdf
+480
View File
@@ -0,0 +1,480 @@
<?xml version="1.0" ?>
<!-- =================================================================================== -->
<!-- | This document was autogenerated by xacro from manipulator2d.xacro | -->
<!-- | EDITING THIS FILE BY HAND IS NOT RECOMMENDED | -->
<!-- =================================================================================== -->
<!-- Manipulator 2D robot (based on Revolute-Revolute Manipulator) -->
<robot name="manipulator2d" xmlns:xacro="http://www.ros.org/wiki/xacro">
<!-- ros_control plugin -->
<gazebo>
<plugin filename="libgazebo_ros_control.so" name="gazebo_ros_control">
<robotNamespace>/manipulator2d</robotNamespace>
<robotSimType>gazebo_ros_control/DefaultRobotHWSim</robotSimType>
</plugin>
</gazebo>
<!-- Links | Colors -->
<gazebo reference="base_link">
<!--<gravity>0</gravity>-->
<material>Gazebo/Black</material>
</gazebo>
<gazebo reference="coupler0">
<!--<gravity>0</gravity>-->
<material>Gazebo/White</material>
</gazebo>
<gazebo reference="link1">
<!--<gravity>0</gravity>-->
<mu1>0.2</mu1>
<mu2>0.2</mu2>
<material>Gazebo/Blue</material>
</gazebo>
<gazebo reference="coupler1">
<!--<gravity>0</gravity>-->
<material>Gazebo/White</material>
</gazebo>
<gazebo reference="link2">
<!--<gravity>0</gravity>-->
<mu1>0.2</mu1>
<mu2>0.2</mu2>
<material>Gazebo/Blue</material>
</gazebo>
<gazebo reference="coupler2">
<material>Gazebo/White</material>
</gazebo>
<gazebo reference="link3">
<!--<gravity>0</gravity>-->
<mu1>0.2</mu1>
<mu2>0.2</mu2>
<material>Gazebo/Blue</material>
</gazebo>
<gazebo reference="camera_link">
<!--<gravity>0</gravity>-->
<mu1>0.2</mu1>
<mu1>0.2</mu1>
<mu2>0.2</mu2>
<material>Gazebo/Red</material>
</gazebo>
<!--&lt;!&ndash; hokuyo &ndash;&gt;-->
<!--<gazebo reference="hokuyo_link">-->
<!--<sensor type="gpu_ray" name="head_hokuyo_sensor">-->
<!--<pose>0 0 0 0 0 0</pose>-->
<!--<visualize>false</visualize>-->
<!--<update_rate>40</update_rate>-->
<!--<ray>-->
<!--<scan>-->
<!--<horizontal>-->
<!--<samples>720</samples>-->
<!--<resolution>1</resolution>-->
<!--<min_angle>-1.570796</min_angle>-->
<!--<max_angle>1.570796</max_angle>-->
<!--</horizontal>-->
<!--</scan>-->
<!--<range>-->
<!--<min>0.10</min>-->
<!--<max>30.0</max>-->
<!--<resolution>0.01</resolution>-->
<!--</range>-->
<!--<noise>-->
<!--<type>gaussian</type>-->
<!--&lt;!&ndash; Noise parameters based on published spec for Hokuyo laser-->
<!--achieving "+-30mm" accuracy at range < 10m. A mean of 0.0m and-->
<!--stddev of 0.01m will put 99.7% of samples within 0.03m of the true-->
<!--reading. &ndash;&gt;-->
<!--<mean>0.0</mean>-->
<!--<stddev>0.01</stddev>-->
<!--</noise>-->
<!--</ray>-->
<!--<plugin name="gazebo_ros_head_hokuyo_controller" filename="libgazebo_ros_gpu_laser.so">-->
<!--<topicName>/manipulator2d/laser/scan</topicName>-->
<!--<frameName>hokuyo_link</frameName>-->
<!--</plugin>-->
<!--</sensor>-->
<!--</gazebo>-->
<!-- camera -->
<gazebo reference="camera_link">
<sensor name="camera1" type="camera">
<update_rate>30.0</update_rate>
<camera name="head">
<horizontal_fov>1.3962634</horizontal_fov>
<image>
<width>800</width>
<height>800</height>
<format>R8G8B8</format>
</image>
<clip>
<near>0.02</near>
<far>300</far>
</clip>
<noise>
<type>gaussian</type>
<!-- Noise is sampled independently per pixel on each frame.
That pixel's noise value is added to each of its color
channels, which at that point lie in the range [0,1]. -->
<mean>0.0</mean>
<stddev>0.007</stddev>
</noise>
</camera>
<plugin filename="libgazebo_ros_camera.so" name="camera_controller">
<alwaysOn>true</alwaysOn>
<updateRate>0.0</updateRate>
<cameraName>manipulator2d/camera1</cameraName>
<imageTopicName>image_raw</imageTopicName>
<cameraInfoTopicName>camera_info</cameraInfoTopicName>
<frameName>camera_link_optical</frameName>
<!-- setting hackBaseline to anything but 0.0 will cause a misalignment
between the gazebo sensor image and the frame it is supposed to
be attached to -->
<hackBaseline>0.0</hackBaseline>
<distortionK1>0.0</distortionK1>
<distortionK2>0.0</distortionK2>
<distortionK3>0.0</distortionK3>
<distortionT1>0.0</distortionT1>
<distortionT2>0.0</distortionT2>
<CxPrime>0</CxPrime>
<Cx>0.0</Cx>
<Cy>0.0</Cy>
<focalLength>0.0</focalLength>
</plugin>
</sensor>
</gazebo>
<material name="black">
<color rgba="0.0 0.0 0.0 1.0"/>
</material>
<material name="blue">
<color rgba="0.0 0.0 0.8 1.0"/>
</material>
<material name="green">
<color rgba="0.0 0.8 0.0 1.0"/>
</material>
<material name="grey">
<color rgba="0.2 0.2 0.2 1.0"/>
</material>
<material name="orange">
<color rgba="1.0 0.423529411765 0.0392156862745 1.0"/>
</material>
<material name="brown">
<color rgba="0.870588235294 0.811764705882 0.764705882353 1.0"/>
</material>
<material name="red">
<color rgba="0.8 0.0 0.0 1.0"/>
</material>
<material name="white">
<color rgba="1.0 1.0 1.0 1.0"/>
</material>
<!-- Used for fixing robot to Gazebo 'base_link' -->
<link name="world"/>
<!--<joint name="fixed" type="fixed">-->
<!--<parent link="world"/>-->
<!--<child link="link1"/>-->
<!--<origin xyz="0 0 ${base_z}" rpy="0 0 0"/>-->
<!--</joint>-->
<joint name="fixed" type="fixed">
<parent link="world"/>
<child link="base_link"/>
<origin rpy="0 0 0" xyz="0 0 0.0"/>
</joint>
<link name="base_link">
<collision>
<origin rpy="0 0 0" xyz="0 0 0.05"/>
<geometry>
<cylinder length="0.1" radius="0.01"/>
</geometry>
</collision>
<visual>
<origin rpy="0 0 0" xyz="0 0 0.05"/>
<geometry>
<cylinder length="0.1" radius="0.01"/>
</geometry>
<material name="black"/>
</visual>
<inertial>
<origin rpy="0 0 0" xyz="0 0 0.05"/>
<mass value="4.93230046614"/>
<inertia ixx="0.0042335579001" ixy="0.0" ixz="0.0" iyy="0.0042335579001" iyz="0.0" izz="0.000246615023307"/>
</inertial>
</link>
<joint name="joint0" type="continuous">
<parent link="base_link"/>
<child link="coupler0"/>
<origin rpy="0 0 0" xyz="0 0 0.075"/>
<axis xyz="0 0 1"/>
<dynamics damping="0.7"/>
</joint>
<link name="coupler0">
<visual>
<origin rpy="0 0 0" xyz="0 0 0"/>
<geometry>
<!--<sphere radius="${radius_axel}"/>-->
<cylinder length="0.05" radius="0.045"/>
</geometry>
<material name="white"/>
</visual>
</link>
<joint name="coupler_joint0" type="fixed">
<parent link="coupler0"/>
<child link="link1"/>
<origin rpy="0 0 0" xyz="0 0 0"/>
</joint>
<!-- Base Link -->
<link name="link1">
<collision>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.15 0 0"/>
<geometry>
<cylinder length="0.3" radius="0.025"/>
</geometry>
</collision>
<visual>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.15 0 0"/>
<geometry>
<cylinder length="0.3" radius="0.025"/>
</geometry>
<material name="blue"/>
</visual>
<inertial>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.15 0 0"/>
<mass value="1.2723450247"/>
<inertia ixx="0.00974139159539" ixy="0.0" ixz="0.0" iyy="0.00974139159539" iyz="0.0" izz="0.00039760782022"/>
</inertial>
</link>
<!--<joint name="joint1" type="continuous">-->
<!--<parent link="link1"/>-->
<!--<child link="link2"/>-->
<!--<origin xyz="${height1 + axel_offset} 0 0" rpy="0 0 0"/>-->
<!--<axis xyz="0 0 1"/>-->
<!--<dynamics damping="0.7"/>-->
<!--</joint>-->
<joint name="joint1" type="continuous">
<parent link="link1"/>
<child link="coupler1"/>
<origin rpy="0 0 0" xyz="0.335 0 0"/>
<axis xyz="0 0 1"/>
<dynamics damping="0.7"/>
</joint>
<link name="coupler1">
<visual>
<origin rpy="0 0 0" xyz="0 0 0"/>
<geometry>
<cylinder length="0.05" radius="0.045"/>
</geometry>
<material name="white"/>
</visual>
</link>
<joint name="coupler_joint1" type="fixed">
<parent link="coupler1"/>
<child link="link2"/>
<origin rpy="0 0 0" xyz="0.035 0 0"/>
</joint>
<!--Middle Link-->
<link name="link2">
<collision>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.1 0 0"/>
<geometry>
<cylinder length="0.2" radius="0.025"/>
</geometry>
</collision>
<visual>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.1 0 0"/>
<geometry>
<cylinder length="0.2" radius="0.025"/>
</geometry>
<material name="blue"/>
</visual>
<inertial>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.1 0 0"/>
<mass value="0.848230016469"/>
<inertia ixx="0.00437368602242" ixy="0.0" ixz="0.0" iyy="0.00437368602242" iyz="0.0" izz="0.000265071880147"/>
</inertial>
</link>
<!--<joint name="joint2" type="continuous">-->
<!--<parent link="link2"/>-->
<!--<child link="link3"/>-->
<!--<origin xyz="${height2 + axel_offset*2} 0.1 0.1" rpy="0 0 0"/>-->
<!--<axis xyz="0 0 1"/>-->
<!--<dynamics damping="0.7"/>-->
<!--</joint>-->
<joint name="joint2" type="continuous">
<parent link="link2"/>
<child link="coupler2"/>
<origin rpy="0 0 0" xyz="0.235 0 0"/>
<axis xyz="0 0 1"/>
<dynamics damping="0.7"/>
</joint>
<link name="coupler2">
<visual>
<origin rpy="0 0 0" xyz="0 0 0"/>
<geometry>
<cylinder length="0.05" radius="0.045"/>
</geometry>
<material name="white"/>
</visual>
</link>
<joint name="coupler_joint2" type="fixed">
<parent link="coupler2"/>
<child link="link3"/>
<origin rpy="0 0 0" xyz="0.035 0 0"/>
</joint>
<!--Top Link-->
<link name="link3">
<collision>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.05 0 0"/>
<geometry>
<cylinder length="0.1" radius="0.025"/>
</geometry>
</collision>
<visual>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.05 0 0"/>
<geometry>
<cylinder length="0.1" radius="0.025"/>
</geometry>
<material name="blue"/>
</visual>
<inertial>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.05 0 0"/>
<mass value="0.424115008235"/>
<inertia ixx="0.00112655549062" ixy="0.0" ixz="0.0" iyy="0.00112655549062" iyz="0.0" izz="0.000132535940073"/>
</inertial>
</link>
<joint name="gripper_joint" type="fixed">
<parent link="link3"/>
<child link="gripper"/>
<origin rpy="0 0 0" xyz="0.1 0 0"/>
</joint>
<link name="gripper">
<visual>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.0075 0 0"/>
<geometry>
<box size="0.14 0.025 0.015"/>
</geometry>
<material name="white"/>
</visual>
<visual>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.0575 0.06 0"/>
<geometry>
<box size="0.02 0.025 0.085"/>
</geometry>
<material name="white"/>
</visual>
<visual>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.0575 -0.06 0"/>
<geometry>
<box size="0.02 0.025 0.085"/>
</geometry>
<material name="white"/>
</visual>
<collision>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.0075 0 0"/>
<geometry>
<box size="0.14 0.025 0.015"/>
</geometry>
<material name="white"/>
</collision>
<collision>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.0575 0.06 0"/>
<geometry>
<box size="0.02 0.025 0.085"/>
</geometry>
<material name="white"/>
</collision>
<collision>
<origin rpy="1.57079632679 0 1.57079632679" xyz="0.0575 -0.06 0"/>
<geometry>
<box size="0.02 0.025 0.085"/>
</geometry>
<material name="white"/>
</collision>
</link>
<!--<joint name="hokuyo_joint" type="fixed">-->
<!--<axis xyz="0 1 0" />-->
<!--<origin xyz="0 0 ${height3 - axel_offset/2}" rpy="0 0 0"/>-->
<!--<parent link="link3"/>-->
<!--<child link="hokuyo_link"/>-->
<!--</joint>-->
<!--&lt;!&ndash; Hokuyo Laser &ndash;&gt;-->
<!--<link name="hokuyo_link">-->
<!--<collision>-->
<!--<origin xyz="0 0 0" rpy="0 0 0"/>-->
<!--<geometry>-->
<!--<box size="0.1 0.1 0.1"/>-->
<!--</geometry>-->
<!--</collision>-->
<!--<visual>-->
<!--<origin xyz="0 0 0" rpy="0 0 0"/>-->
<!--<geometry>-->
<!--<mesh filename="package://manipulator2d_description/meshes/hokuyo.dae"/>-->
<!--</geometry>-->
<!--</visual>-->
<!--<inertial>-->
<!--<mass value="1e-5" />-->
<!--<origin xyz="0 0 0" rpy="0 0 0"/>-->
<!--<inertia ixx="1e-6" ixy="0" ixz="0" iyy="1e-6" iyz="0" izz="1e-6" />-->
<!--</inertial>-->
<!--</link>-->
<joint name="camera_joint" type="fixed">
<axis xyz="0 1 0"/>
<origin rpy="0 1.57079632679 1.57079632679" xyz="0.0 0.0 1.5"/>
<parent link="base_link"/>
<child link="camera_link"/>
</joint>
<!-- Camera -->
<link name="camera_link">
<!--<collision>-->
<!--<origin xyz="0 0 0" rpy="0 0 0"/>-->
<!--<geometry>-->
<!--<box size="${camera_link} ${camera_link} ${camera_link}"/>-->
<!--</geometry>-->
<!--</collision>-->
<!--<visual>-->
<!--<origin xyz="0 0 0" rpy="0 0 0"/>-->
<!--<geometry>-->
<!--<box size="${camera_link} ${camera_link} ${camera_link}"/>-->
<!--</geometry>-->
<!--<material name="red"/>-->
<!--</visual>-->
<!--<inertial>-->
<!--<mass value="1e-5" />-->
<!--<origin xyz="0 0 0" rpy="0 0 0"/>-->
<!--<inertia ixx="1e-6" ixy="0" ixz="0" iyy="1e-6" iyz="0" izz="1e-6" />-->
<!--</inertial>-->
</link>
<!-- generate an optical frame http://www.ros.org/reps/rep-0103.html#suffix-frames
so that ros and opencv can operate on the camera frame correctly -->
<joint name="camera_optical_joint" type="fixed">
<!-- these values have to be these values otherwise the gazebo camera image
won't be aligned properly with the frame it is supposedly originating from -->
<origin rpy="-1.57079632679 0 -1.57079632679" xyz="0 0 0"/>
<parent link="camera_link"/>
<child link="camera_link_optical"/>
</joint>
<link name="camera_link_optical">
</link>
<transmission name="tran0">
<type>transmission_interface/SimpleTransmission</type>
<joint name="joint0">
<hardwareInterface>hardware_interface/EffortJointInterface</hardwareInterface>
</joint>
<actuator name="motor0">
<hardwareInterface>hardware_interface/EffortJointInterface</hardwareInterface>
<mechanicalReduction>1</mechanicalReduction>
</actuator>
</transmission>
<transmission name="tran1">
<type>transmission_interface/SimpleTransmission</type>
<joint name="joint1">
<hardwareInterface>hardware_interface/EffortJointInterface</hardwareInterface>
</joint>
<actuator name="motor1">
<hardwareInterface>hardware_interface/EffortJointInterface</hardwareInterface>
<mechanicalReduction>1</mechanicalReduction>
</actuator>
</transmission>
<transmission name="tran2">
<type>transmission_interface/SimpleTransmission</type>
<joint name="joint2">
<hardwareInterface>hardware_interface/EffortJointInterface</hardwareInterface>
</joint>
<actuator name="motor2">
<hardwareInterface>hardware_interface/EffortJointInterface</hardwareInterface>
<mechanicalReduction>1</mechanicalReduction>
</actuator>
</transmission>
</robot>
pyrobolearn/simulators/README.md
+11 -1
View File
@@ -1,3 +1,13 @@
## Simulators
This folder contains the APIs to the various simulators. Currently, the main simulator being supported is PyBullet. Work is under progress for Gazebo+ROS, and OpenSIM.
This folder contains the APIs to the various simulators. Currently, the main simulator being supported is PyBullet. Work is under progress for Gazebo+ROS, and OpenSIM.
```python
import pyrobolearn as prl
sim = prl.simulators.BulletSim()
```
#### What to check next?
Check the `worlds` folder and the `robots` folder.
pyrobolearn/states/README.md
+26 -5
View File
@@ -1,11 +1,32 @@
## States
The `State` is returned by the environment and given to the policy. The state might include information about the state of one or several objects in the world, including robots.
The `State` is returned by the environment and given to the policy. The state might include information about the state of one or several objects in the world, including robots.
It is the main bridge between the robots/objects in the environment and the policy. Specifically, it is given as an input to the policy which knows how to feed the state to the learning model. Usually, the user only has to instantiate a child of this class, and give it to the policy and environment, and that's it. In addition to the policy, the state can be given to a controller, dynamic model, value estimator, reward function, and so on.
To allow the framework to be modular, we favor composition over inheritance [1] leading the state to be decoupled from notions such as the environment, policy, rewards, etc. This class also describes the `state_space` which has initially been defined in `gym.Env` [2].
To allow the framework to be modular and flexible, we favor [composition over inheritance][https://en.wikipedia.org/wiki/Composition_over_inheritance] leading the state to be decoupled from notions such as the environment, policy, rewards, etc. This class also describes the `state_space` which has initially been defined in [`gym.Env`](https://github.com/openai/gym).
References:
[1] "Wikipedia: Composition over Inheritance", https://en.wikipedia.org/wiki/Composition_over_inheritance
[2] "OpenAI gym": https://gym.openai.com/ and https://github.com/openai/gym
```python
from itertools import count
import time
import pyrobolearn as prl
sim = prl.simulators.BulletSim()
world = prl.worlds.BasicWorld(sim)
robot = prl.robots.loadRobot('wam') # try another robot such as 'coman' or 'littledog'
state = prl.states.JointPositionState(robot) + prl.states.JointVelocityState(robot) # you can add other states
env = prl.envs.Env(world, state) # the state, world, and possible rewards are defined outside the environment
for t in count():
if (t % 240) == 0:
print(state) # print the joint positions and velocities of the specified robot
env.step() # this will ask the state to read or compute the next value, and perform a step in the simulator
time.sleep(1./240)
```
#### What to check/look next?
Check first the `actions` folder, then the `approximators`, `policies`, `rewards`, and `envs` folders.
pyrobolearn/tasks/README.md
+13
View File
@@ -0,0 +1,13 @@
## Learning Tasks
This folder contains learning tasks and how they should be run. These include:
- Reinforcement learning tasks: the task accepts as inputs the environment and the policies. This can then be given to a RL algorithm.
- Imitation learning tasks: the task accepts as inputs the environment, the policies, and the bridges to the interface used to interact with a part of the simulator (robot, object in the world, etc).
Other learning paradigms include (but are not implemented yet):
- Active learning
- Curriculum learning
- Inverse reinforcement learning
- Transfer learning
Each task can be evaluated using different metrics.
pyrobolearn/tools/README.md
+8 -1
View File
@@ -5,4 +5,11 @@ This folder the *interfaces* and the *bridges*.
* The I/O *interfaces* get (or set) the data from (to) the hardware, process it, and store it inside the class.
* The *bridges* makes the connection between the interface and a component (such as the world or an element in that world such as a robot) in the framework.
The separation between interfaces and bridges allows for better flexibility. For instance, a game controller interface allows us to get data from the hardware, process it, and store it inside the class. The bridge can then map the specific controller events to a robot. Moving a joystick up could mean to move a UAV robot up in the air, or move a wheeled robot forward.
The separation between interfaces and bridges allows for better flexibility. For instance, a game controller interface allows us to get data from the hardware, process it, and store it inside the class. The bridge can then map the specific controller events to a robot. Moving a joystick up could mean to move a UAV robot up in the air, or move a wheeled robot forward.
While few bridges are already provided in the framework, most of them are let to the user to implement them.
TODO:
- [ ] clean few interfaces
- [ ] provide other interfaces (FER, google assistant and/or alexa, etc.)
- [ ] provide different bridges
pyrobolearn/utils/README.md
+3
View File
@@ -0,0 +1,3 @@
## Utils
This folder contains several utils methods/classes that are generally useful.
pyrobolearn/worlds/README.md
+22
View File
@@ -0,0 +1,22 @@
## Worlds
In this folder, we provide the `World` class, and the `BasicWorld` class (which loads the floor and sets the gravity).
With this world, you can load floors, visual and collision objects, robots, and others. The world is with the robot (+ actuators/sensors) and the mouse keyboard interface, the only parts in the framework that can interact with the simulator directly. You can load a robot through the world.
```python
import pyrobolearn as prl
sim = prl.simulators.BulletSim()
world = prl.worlds.BasicWorld(sim)
coman = world.loadRobot('Coman', position=<position1>)
wam = world.loadRobot(prl.robots.WAM, position=<position2>)
# the following is not advised
littledog = prl.robots.LittleDog(sim, init_pos=<position3>)
world.loadRobot(littledog) # such that the world knows about the robot is present.
```
#### What to check/look next?
Check the `robots` folder.