deepr.prepros.TableInitializer

class deepr.prepros.TableInitializer(table_initializer_fn)

Table Initializer.

Tensorflow does not allow tables initialization inside a map transformation (all tables must be created outside the map).

To remedy this, follow this example

First, create a table_initializer_fn that uses the tf.AUTO_REUSE argument.

>>> import deepr
>>> def table_initializer_fn():
...     return deepr.utils.table_from_mapping(
...         name="partner_table", mapping={1: 2}, reuse=tf.AUTO_REUSE
... )

Then, define your preprocessing pipeline as follows

>>> prepro_fn = deepr.prepros.Serial(
...     deepr.prepros.TableInitializer(table_initializer_fn),
...     deepr.prepros.Map(deepr.layers.Lookup(table_initializer_fn)),
... )

When applying the prepro_fn on a tf.data.Dataset, it will run the table_initializer_fn at the beginning (outside the map transformation), then apply the Lookup that uses the same table_initializer_fn, but thanks to reuse=tf.AUTO_REUSE instead of creating a new table, it will simply reuse the table created by the TableInitializer.

__init__(table_initializer_fn)

Methods

__init__(table_initializer_fn)
apply(dataset[, mode])	Pre-process a dataset