Spaces

Spaces are used to specify the interfaces from the environment to the agent (model); the describe the observations and actions.

class rlpyt.spaces.base.Space

Common definitions for observations and actions.

sample()

Uniformly randomly sample a random element of this space.

null_value()

Return a null value used to fill for absence of element.

class rlpyt.spaces.int_box.IntBox(low, high, shape=None, dtype='int32', null_value=None)

Bases: rlpyt.spaces.base.Space

A box in J^n, with specificiable bound and dtype.

__init__(low, high, shape=None, dtype='int32', null_value=None)

Params low and high are scalars, applied across all dimensions of shape; valid values will be those in range(low, high).

sample()

Return a single sample from np.random.randint.

n

The number of elements in the space.

class rlpyt.spaces.float_box.FloatBox(low, high, shape=None, null_value=0.0, dtype='float32')

Bases: rlpyt.spaces.base.Space

A box in R^n, with specifiable bounds and dtype.

__init__(low, high, shape=None, null_value=0.0, dtype='float32')

Two kinds of valid input:

• low and high are scalars, and shape is provided: Box(-1.0, 1.0, (3,4))
• low and high are arrays of the same shape: Box(np.array([-1.0,-2.0]), np.array([2.0,4.0]))

sample()

Return a single sample from np.random.uniform.

class rlpyt.spaces.composite.Composite(spaces, NamedTupleCls)

Bases: rlpyt.spaces.base.Space

A space for composing arbitrary combinations of spaces together.

__init__(spaces, NamedTupleCls)

Must input the instantiated sub-spaces in order (e.g. list or tuple), and a named tuple class with whch to organize the sub-spaces and resulting samples. The NamedTupleCls should be defined in the module (file) which defines the composite space.

sample()

Return a single sample which is a named tuple composed of samples from all sub-spaces.

null_value()

Return a null value which is a named tuple composed of null values from all sub-spaces.

shape

Return a named tuple composed of shapes of every sub-space.

names

Return names of sub-spaces.

spaces

Return the bare sub-spaces.

class rlpyt.spaces.gym_wrapper.GymSpaceWrapper(space, null_value=0, name='obs', force_float32=True)

Wraps a gym space to match the rlpyt interface; most of the functionality is for automatically converting a GymDict (dictionary) space into an rlpyt Composite space (and converting between the two). Use inside the initialization of the environment wrapper for a gym environment.

__init__(space, null_value=0, name='obs', force_float32=True)

Input space is a gym space instance.

Input name is used to disambiguate different gym spaces being wrapped, which is necessary if more than one GymDict space is to be wrapped in the same file. The reason is that the associated namedtuples must be defined in the globals of this file, so they must have distinct names.

sample()

Returns a single sample in a namedtuple (for composite) or numpy array using the the sample() method of the underlying gym space(s).

null_value()

Similar to sample() but returning a null value.

convert(value)

For dictionary space, use to convert wrapped env’s dict to rlpyt namedtuple, i.e. inside the environment wrapper’s step(), for observation output to the rlpyt sampler (see helper function in file)

revert(value)

For dictionary space, use to revert namedtuple action into wrapped env’s dict, i.e. inside the environment wrappers step(), for input to the underlying gym environment (see helper function in file).