Changelog
=========
All notable changes to this project will be documented in this file.
The format is based on `Keep a Changelog `_
and this project adheres to `Semantic Versioning `_.
[2.13.0] - 2023-01-07
---------------------
Added
~~~~~
Changed
~~~~~~~
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
- Fixed CI to make it keep passing on Python 3.6
- Fix mlflow compatibility with mlflow server 1.30.0
Security
~~~~~~~~
[2.12.0] - 2021-04-29
---------------------
Added
~~~~~
- function support for ``Pipeline``
Changed
~~~~~~~
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
Security
~~~~~~~~
[2.11.0] - 2020-12-11
---------------------
Added
~~~~~
- Add new name for ``layers.Sequential`` (now ``layers.DAG`` for Directed Acyclic Graph), kept old name for legacy reasons
- Add new argument options to ``layers.DenseIndex`` (reuse, trainable and initializer)
Changed
~~~~~~~
- Remove shortened imports ``import deepr as dpr`` as it is useless.
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
- Unnecessary logging in metrics tensors' names resolution
Fixed
~~~~~
- Incorrect numpy requirement (incompatible with Tensorflow requirements)
Security
~~~~~~~~
[2.10.0] - 2020-10-28
---------------------
Added
~~~~~
- Log yarn launcher app id with mlflow.
[2.9.1] - 2020-10-01
--------------------
Fixed
~~~~~
- move tf-yarn dependency to gpu / cpu additional packages.
[2.9.0] - 2020-10-01
--------------------
Added
~~~~~
- SVD jobs and configs to MovieLens
- VAE model to MovieLens
- NDCG@K and Recall@K metrics to MovieLens
- CSVReader to MovieLens
- AddWithWeight, DenseIndex, MultiLogLikelihoodCSS, MultiLogLikelihood layers
- Lambda layer to layers.base
- preds option to Trainer
Changed
~~~~~~~
[2.8.2] - 2020-09-07
--------------------
Fixed
~~~~~
- Convert tuple of FileSystems into a List for skein (not cast by cluster_pack).
[2.8.1] - 2020-09-03
--------------------
Fixed
~~~~~
- breaking change in grad norms in Optimizer (store by query name, not variable name)
[2.8.0] - 2020-09-03
--------------------
Added
~~~~~
- Accuracy and AccuracyAtK metrics
- EvaluateJob
- LSTM layer (using FusedOp)
- Recursive .glob() in TFRecordReader for nested directory structures
- Experimental Keras Trainer (converts tf.keras.Model into tf.estimator.Estimator)
- Add run_id parameter to MlFlow macro (to restart a run)
- Refactor to_example logic (make ``arrays_to_example`` importable to convert NumPy arrays to tf.Example)
- Add missing macros module import
- Add batch_shape to Field
Changed
~~~~~~~
- ``__iter__`` method of `Reader` (remove context manager to avoid issue with Keras)
- use skein_launcher instead of in-house implementation using skein.
Fixed
~~~~~
- Use skip_steps when computing gradient norms in Optimizer
- Path copy_file (to support local <> HDFS copies)
[2.7.0] - 2020-08-04
--------------------
Added
~~~~~
- Add encoding support to Path / HDFSFile
- Movielens example
- TripletPrecision layer
Changed
~~~~~~~
- Changed default initializer for embeddings in the embeddings layer
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
Security
~~~~~~~~
[2.6.0] - 2020-07-02
--------------------
Added
~~~~~
- Add table support in ProtoPredict and ProtoExport
[2.5.1] - 2020-07-01
--------------------
Fixed
~~~~~
- wrong attribute in CopyDir
[2.5.0] - 2020-07-01
--------------------
Added
~~~~~
- Batched support for ``FromExample`` (use ``tf.train.parse_example`` instead of ``parse_single_example``)
- Support ``num_shards_embeddings=None`` in ``utils.save_checkpoint``
- Top1, Top1Max, BPRMax, NCE losses
Changed
~~~~~~~
- Various versions of requirements in ``requirements.txt``
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
Security
~~~~~~~~
[2.4.2] - 2020-06-19
--------------------
Added
~~~~~
- Allow user to ignore cpu packages shipped in the pex to Yarn
- URL formatter for MLFlow Macro
- More checks to ``vocab.write`` (check type, newlines)
[2.4.1] - 2020-06-15
--------------------
Fixed
~~~~~
- Fix incorrect inputs / outputs resolution in ``deepr.layers.Select``
- Fix edge cases of ``ToExample`` and ``FromExample`` (on scalars, tensors with dynamic shapes with ndims > 2)
[2.4.0] - 2020-06-10
--------------------
Added
~~~~~
- Utilities to ``metrics.base``: ``get_tensors``, ``keep_scalars``, ``get_scalars``
- New metrics for variables ``VariableValue`` that returns value / global norm of a variable
Changed
~~~~~~~
- ``LastValue`` metric does not store tensor values in a special metric variable.
[2.3.0] - 2020-06-10
--------------------
Added
~~~~~
- ``writers`` module, ``TFRecordWriter``
- ``ToExample`` prepro to convert a dataset to ``tf.Example``
- ``Field`` now has a ``to_feature(value)`` method
- ``iter`` utils: ``progress`` (logs progress every n seconds), ``chunks`` to return chunks from an iterable
- ``SaveDataset`` job to write a dataset to tfrecords.
Changed
~~~~~~~
- ``TFRecordSequenceExample`` renamed ``FromExample`` (but old name still available).
- ``Field`` method ``as_feature`` renamed ``feature_specs`` to avoid confusion with ``to_feature``.
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
- Removed ``Field`` methods (leading to incorrect uses): ``has_var_len``, ``as_feature``, ``has_fixed_len``
Fixed
~~~~~
- Incorrect ``shuffle`` argument use in ``TFRecordReader``
Security
~~~~~~~~
[2.2.0] - 2020-06-08
------------
Added
~~~~~
ExportXlaModelMetadata job is added : make it possible to export metadata for xla models
Changed
~~~~~~~
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
Security
~~~~~~~~
[2.1.1] - 2020-06-05
--------------------
Added
~~~~~
- Predictors also yield inputs when applied on a ``tf.data.Dataset``
Changed
~~~~~~~
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
Security
~~~~~~~~
[2.1.0] - 2020-06-04
--------------------
Added
~~~~~
- Add ``predictors``
- Add new example job ``PredictSavedModel``
Changed
~~~~~~~
- Example job ``Predict`` renamed into ``PredictProto``
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
Security
~~~~~~~~
[2.0.0] - 2020-06-03
--------------------
Added
~~~~~
- Doctest run in CI.
Changed
~~~~~~~
- copy_dir job will now overwrite the target by default
- Nested support for ``prepros.Serial``
- Context manager ``TableContext`` for tables reuse
- Automatic table context creation in ``prepro.__call__``
- Prepro ``TableInitializer`` to run ``table_initializer_fn`` before ``map`` transforms
- Vocabulary utilities (``read``, ``write``, ``size``)
- Reverse lookup table function ``index_to_string_table_from_file`` and associated layer ``LookupIndexToString``
- Layer combinator ``ActiveMode`` to apply layer only on given modes
- Layer ``ToFloat``
- Config evaluation modes: ``skip`` -> ``None``, ``instance`` -> ``call``
- New evaluation mode for config dictionary ``partial``
- Remove ``__post_init__`` for ``YarnTrainer`` and ``YarnLauncher`` to avoid unexpected non-laziness
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
- Use of ``prepro`` and ``layer`` decorator on constructors
- Lazy behavior for ``prepro`` and ``layer`` decorator
Fixed
~~~~~
- Doctests were fixed.
- Add metric name sanitizer, especially needed to sanitize keras built variable names.
- Typo in ``example`` predict (feedable / fetchable)
Security
~~~~~~~~
[1.2.1] - 2020-05-27
--------------------
Added
~~~~~
Changed
~~~~~~~
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
Avoid mkdir for HDFS path_model for permissions reasons
Security
~~~~~~~~
[1.2.0] - 2020-05-26
--------------------
Added
~~~~~
- ``OptimizeSavedModel`` now supports multiple fetches
- new graph utils, ``import_graph_def``, ``get_feedable_tensors``, ``get_fetchable_tensors``
Changed
~~~~~~~
- ``example.jobs.Predict`` arguments (``path_model`` and ``graph_name`` instead of ``path_model_pb``, ``fetch`` instead of ``fetches`` for consistency with ``OptimizeSavedModel``).
Deprecated
~~~~~~~~~~
Removed
~~~~~~~
Fixed
~~~~~
- wrong arguments in ``YarnConfig`` for ``upload_zip_to_hdfs``.
Security
~~~~~~~~
[1.1.0] - 2020-05-25
--------------------
Added
~~~~~
- Remove some kwargs for cleaner error stacks
- Make example more complex, add advanced notebook
- Track missing macro
- Update doc of logging tensor (change prefix to name)
- Add helper to debug class building from config
[1.0.0] - 2020-05-19
--------------------
Added
~~~~~
- Public Release