merlion.models package
Broadly, Merlion contains two types of models: anomaly detection (merlion.models.anomaly
)
and forecasting (merlion.models.forecast
). Note that there is a distinct subset of anomaly
detection models that use forecasting models at their core (merlion.models.anomaly.forecast_based
).
We implement an abstract ModelBase
class which provides the following functionality for all models:
model = ModelClass(config)
initialization with a model-specific config (which inherits from
Config
)configs contain:
a (potentially trainable) data pre-processing transform from
merlion.transform
; note thatmodel.transform
is a property which refers tomodel.config.transform
model-specific hyperparameters
model.save(dirname, save_config=None)
saves the model to the specified directory. The model’s configuration is saved to
<dirname>/config.json
, while the model’s binary data is (by default) saved in binary form to<dirname>/model.pkl
. Note that if you edit the saved<dirname>/config.json
on disk, the changes will be loaded when you callModelClass.load(dirname)
!this method heavily exploits the fact that many objects in Merlion are JSON-serializable
ModelClass.load(dirname, **kwargs)
this class method initializes an instance of
ModelClass
from the config file saved in<dirname>/config.json
, (overriding any parameters of the config withkwargs
where relevant), loads the remaining binary data into the model object, and returns the fully initialized model.
For users who aren’t familiar with the specific details of various models, we provide default models for anomaly
detection and forecasting in merlion.models.defaults
.
We also provide a ModelFactory
which can be used to conveniently instantiate models from their name and a set of
keyword arguments, or to load them directly from disk. For example, we may have the following workflow:
from merlion.models.factory import ModelFactory
from merlion.models.anomaly.windstats import WindStats, WindStatsConfig
# creates the same kind of model in 2 equivalent ways
model1a = WindStats(WindStatsConfig(wind_sz=60))
model1b = ModelFactory.create("WindStats", wind_sz=60)
# save the model & load it in 2 equivalent ways
model1a.save("tmp")
model2a = WindStats.load("tmp")
model2b = ModelFactory.load("tmp")
Finally, we support ensembles of models in merlion.models.ensemble
.
Default models for anomaly detection & forecasting that balance speed and performance. |
|
Contains the |
|
Contains the base classes for all models. |
|
Contains the base classes for all deep learning models. |
|
Base class for layered models. |
|
Contains all anomaly detection models. |
|
Contains all change point detection algorithms. |
|
Contains all forecaster-based anomaly detectors. |
|
Contains all forecasting models, including those which support exogenous regressors. |
|
Ensembles of models and automated model selection. |
|
Contains all AutoML model variants & some utilities. |
|
Contains various utility files & functions useful for different models. |
Subpackages
defaults
Default models for anomaly detection & forecasting that balance speed and performance.
- class merlion.models.defaults.DefaultDetectorConfig(model=None, granularity=None, n_threads=1, model_kwargs=None, transform=None, **kwargs)
Bases:
LayeredModelConfig
Config object for default anomaly detection model.
- Parameters
model – The model being wrapped, or a dict representing it.
granularity (
Optional
[str
]) – the granularity at which the input time series should be sampled, e.g. “5min”, “1h”, “1d”, etc.n_threads (
int
) – the number of parallel threads to use for relevant modelsmodel_kwargs – Keyword arguments used specifically to initialize the underlying model. Only used if
model
is a dict. Will override keys in themodel
dict if specified.transform – Transformation to pre-process input time series.
kwargs – Any other keyword arguments (e.g. for initializing a base class). If
model
is a dict, we will also try to pass these arguments when creating the actual underlying model. However, they will not override arguments in either themodel
dict ormodel_kwargs
dict.
- class merlion.models.defaults.DefaultDetector(config=None, model=None, **kwargs)
Bases:
LayeredDetector
Default anomaly detection model that balances efficiency with performance.
- Parameters
config (
Optional
[LayeredModelConfig
]) – model configuration
- config_class
alias of
DefaultDetectorConfig
- property granularity
- reset()
Resets the model’s internal state.
- train(train_data, train_config=None, anomaly_labels=None, post_rule_train_config=None)
Trains the anomaly detector (unsupervised) and its post-rule (supervised, if labels are given) on train data.
- Parameters
train_data (
TimeSeries
) – aTimeSeries
of metric values to train the model.train_config – Additional training configs, if needed. Only required for some models.
anomaly_labels (
Optional
[TimeSeries
]) – aTimeSeries
indicating which timestamps are anomalous. Optional.post_rule_train_config – The config to use for training the model’s post-rule. The model’s default post-rule train config is used if none is supplied here.
- Return type
- Returns
A
TimeSeries
of the model’s anomaly scores on the training data.
- class merlion.models.defaults.DefaultForecasterConfig(model=None, max_forecast_steps=None, target_seq_index=None, granularity=None, model_kwargs=None, transform=None, **kwargs)
Bases:
LayeredModelConfig
Config object for default forecasting model.
- Parameters
model – The model being wrapped, or a dict representing it.
max_forecast_steps (
Optional
[int
]) – Max # of steps we would like to forecast for.target_seq_index (
Optional
[int
]) – The index of the univariate (amongst all univariates in a general multivariate time series) whose value we would like to forecast.granularity (
Optional
[str
]) – the granularity at which the input time series should be sampled, e.g. “5min”, “1d”, etc.model_kwargs – Keyword arguments used specifically to initialize the underlying model. Only used if
model
is a dict. Will override keys in themodel
dict if specified.transform – Transformation to pre-process input time series.
kwargs – Any other keyword arguments (e.g. for initializing a base class). If
model
is a dict, we will also try to pass these arguments when creating the actual underlying model. However, they will not override arguments in either themodel
dict ormodel_kwargs
dict.
- class merlion.models.defaults.DefaultForecaster(config=None, model=None, **kwargs)
Bases:
LayeredForecaster
Default forecasting model that balances efficiency with performance.
- config_class
alias of
DefaultForecasterConfig
- property supports_exog
Whether the model supports exogenous regressors.
- property granularity
- reset()
Resets the model’s internal state.
- train(train_data, train_config=None, exog_data=None)
Trains the forecaster on the input time series.
- Parameters
train_data (
TimeSeries
) – aTimeSeries
of metric values to train the model.train_config – Additional training configs, if needed. Only required for some models.
exog_data – A time series of exogenous variables, sampled at the same time stamps as
train_data
. Exogenous variables are known a priori, and they are independent of the variable being forecasted. Only supported for models which inherit fromForecasterExogBase
.
- Return type
Tuple
[TimeSeries
,Optional
[TimeSeries
]]- Returns
the model’s prediction on
train_data
, in the same format as if you calledForecasterBase.forecast
on the time stamps oftrain_data
factory
Contains the ModelFactory.
- class merlion.models.factory.ModelFactory
Bases:
object
- classmethod create(name, return_unused_kwargs=False, **kwargs)
- merlion.models.factory.instantiate_or_copy_model(model)
base
Contains the base classes for all models.
- class merlion.models.base.Config(transform=None, **kwargs)
Bases:
object
Abstract class which defines a model config.
- Parameters
transform (
Optional
[TransformBase
]) – Transformation to pre-process input time series.
- filename = 'config.json'
-
transform:
TransformBase
= None
-
dim:
Optional
[int
] = None
- to_dict(_skipped_keys=None)
- Returns
dict with keyword arguments used to initialize the config class.
- classmethod from_dict(config_dict, return_unused_kwargs=False, dim=None, **kwargs)
Constructs a Config from a Python dictionary of parameters.
- Parameters
config_dict (
Dict
[str
,Any
]) – dict that will be used to instantiate this object.return_unused_kwargs – whether to return any unused keyword args.
dim – the dimension of the time series. handled as a special case.
kwargs – any additional parameters to set (overriding config_dict).
- Returns
Config object initialized from the dict.
- get_unused_kwargs(**kwargs)
- class merlion.models.base.NormalizingConfig(normalize=None, transform=None, **kwargs)
Bases:
Config
Model config where the transform must return normalized values. Applies additional normalization after the initial data pre-processing transform.
- Parameters
normalize (
Optional
[Rescale
]) – Pre-trained normalization transformation (optional).transform – Transformation to pre-process input time series.
- property full_transform
Returns the full transform, including the pre-processing step, lags, and final mean/variance normalization.
- property transform
- class merlion.models.base.ModelBase(config)
Bases:
object
Abstract base class for models.
- filename = 'model.pkl'
-
train_data:
Optional
[TimeSeries
] = None The data used to train the model.
- reset()
Resets the model’s internal state.
- property base_model
The base model of a base model is itself.
- abstract property require_even_sampling: bool
Whether the model assumes that training data is sampled at a fixed frequency
- abstract property require_univariate: bool
Whether the model only works with univariate time series.
- property auto_align: bool
Whether to ensure that all univariates in the training data are aligned.
- property supports_exog
Whether the model supports exogenous regressors.
- property dim
- property transform
- Returns
The data pre-processing transform to apply on any time series, before giving it to the model.
- property timedelta
- Returns
the gap (as a
pandas.Timedelta
orpandas.DateOffset
) between data points in the training data
- property last_train_time
- Returns
the last time (as a
pandas.Timestamp
) that the model was trained on
- train_pre_process(train_data)
Applies pre-processing steps common for training most models.
- Parameters
train_data (
TimeSeries
) – the original time series of training data- Return type
- Returns
the training data, after any necessary pre-processing has been applied
- transform_time_series(time_series, time_series_prev=None)
Applies the model’s pre-processing transform to
time_series
andtime_series_prev
.- Parameters
time_series (
TimeSeries
) – The time seriestime_series_prev (
Optional
[TimeSeries
]) – A time series of context, immediately precedingtime_series
. Optional.
- Return type
Tuple
[TimeSeries
,Optional
[TimeSeries
]]- Returns
The transformed
time_series
andtime_series_prev
.
- abstract train(train_data, train_config=None)
Trains the model on the specified time series, optionally with some additional implementation-specific config options
train_config
.- Parameters
train_data (
TimeSeries
) – a TimeSeries to use as a training settrain_config – additional configurations (if needed)
- abstract train_post_process(train_result)
- save(dirname, **save_config)
- Parameters
dirname (
str
) – directory to save the model & its configsave_config – additional configurations (if needed)
- classmethod load(dirname, **kwargs)
- Parameters
dirname (
str
) – directory to load model (and config) fromkwargs – config params to override manually
- Returns
ModelBase object loaded from file
- to_bytes(**save_config)
Converts the entire model state and configuration to a single byte object.
- Returns
bytes object representing the model.
- classmethod from_bytes(obj, **kwargs)
Creates a fully specified model from a byte object
- Parameters
obj – byte object to convert into a model
- Returns
ModelBase object loaded from
obj
- class merlion.models.base.MultipleTimeseriesModelMixin
Bases:
object
Abstract mixin for models supporting training on multiple time series.
- abstract train_multiple(multiple_train_data, train_config=None)
Trains the model on multiple time series, optionally with some additional implementation-specific config options
train_config
.- Parameters
multiple_train_data (
List
[TimeSeries
]) – a list of TimeSeries to use as a training settrain_config – additional configurations (if needed)
deep_base
Contains the base classes for all deep learning models.
- class merlion.models.deep_base.Optimizer(value)
Bases:
Enum
Optimizers for learning model parameters.
- Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False, *, foreach=None, maximize=False, capturable=False, differentiable=False, fused=False) = <class 'torch.optim.adam.Adam'>
- AdamW(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.01, amsgrad=False, *, maximize=False, foreach=None, capturable=False) = <class 'torch.optim.adamw.AdamW'>
- SGD(params, lr=<required parameter>, momentum=0, dampening=0, weight_decay=0, nesterov=False, *, maximize=False, foreach=None, differentiable=False) = <class 'torch.optim.sgd.SGD'>
- Adagrad(params, lr=0.01, lr_decay=0, weight_decay=0, initial_accumulator_value=0, eps=1e-10, foreach=None, *, maximize=False) = <class 'torch.optim.adagrad.Adagrad'>
- RMSprop(params, lr=0.01, alpha=0.99, eps=1e-08, weight_decay=0, momentum=0, centered=False, foreach=None, maximize=False, differentiable=False) = <class 'torch.optim.rmsprop.RMSprop'>
- class merlion.models.deep_base.LossFunction(value)
Bases:
Enum
Loss functions for learning model parameters.
- mse(size_average=None, reduce=None, reduction='mean') = <class 'torch.nn.modules.loss.MSELoss'>
- l1(size_average=None, reduce=None, reduction='mean') = <class 'torch.nn.modules.loss.L1Loss'>
- huber(reduction='mean', delta=1.0) = <class 'torch.nn.modules.loss.HuberLoss'>
- guassian_nll(*, full=False, eps=1e-06, reduction='mean') = <class 'torch.nn.modules.loss.GaussianNLLLoss'>
- class merlion.models.deep_base.DeepConfig(batch_size=32, num_epochs=10, optimizer=Optimizer.Adam, loss_fn=LossFunction.mse, clip_gradient=None, use_gpu=False, ts_encoding='h', lr=0.0001, weight_decay=0.0, valid_fraction=0.2, early_stop_patience=None, **kwargs)
Bases:
Config
Config object used to define a deep learning (pytorch) model.
- Parameters
batch_size (
int
) – Batch size of a batch for stochastic training of deep modelsnum_epochs (
int
) – Total number of epochs for training.optimizer (
Union
[str
,Optimizer
]) – The optimizer for learning the parameters of the deep learning models. The value of optimizer can beAdam
,AdamW
,SGD
,Adagrad
,RMSprop
.loss_fn (
Union
[str
,LossFunction
]) – Loss function for optimizing deep learning models. The value of loss_fn can bemse
for l2 loss,l1
for l1 loss,huber
for huber loss.clip_gradient (
Optional
[float
]) – Clipping gradient norm of model parameters before updating. Ifclip_gradient is None
, then the gradient will not be clipped.use_gpu (
bool
) – Whether to use gpu for training deep models. Ifuse_gpu = True
while thre is no GPU device, the model will use CPU for training instead.ts_encoding (
Optional
[str
]) – whether the timestamp should be encoded to a float vector, which can be used for training deep learning based time series models; ifNone
, the timestamp is not encoded. If notNone
, it represents the frequency for time features encoding options:[s:secondly, t:minutely, h:hourly, d:daily, b:business days, w:weekly, m:monthly]lr (
float
) – Learning rate for optimizing deep learning models.weight_decay (
float
) – Weight decay (L2 penalty) (default: 0)valid_fraction (
float
) – Fraction of validation set to be split from training dataearly_stop_patience (
Optional
[int
]) – Number of epochs with no improvement after which training will be stopped for early stopping function. Ifearly_stop_patience = None
, the training process will not stop early.transform – Transformation to pre-process input time series.
- property loss_fn: LossFunction
- class merlion.models.deep_base.TorchModel(config)
Bases:
Module
Abstract base class for Pytorch deep learning models
Initializes internal Module state, shared by both nn.Module and ScriptModule.
- abstract forward(past, past_timestamp, future_timestamp, *args, **kwargs)
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- property device
- class merlion.models.deep_base.DeepModelBase(config)
Bases:
ModelBase
Abstract base class for all deep learning models
- config_class
alias of
DeepConfig
- deep_model_class
alias of
TorchModel
- to_gpu()
Move deep model to GPU
- to_cpu()
Move deep model to CPU
layers
Base class for layered models. These are models which act as a wrapper around another model, often with additional functionality. This is the basis for default models and AutoML models.
- class merlion.models.layers.LayeredModelConfig(model, model_kwargs=None, transform=None, **kwargs)
Bases:
Config
Config object for a LayeredModel. See LayeredModel documentation for more details.
- Parameters
model (
Union
[ModelBase
,Dict
]) – The model being wrapped, or a dict representing it.model_kwargs – Keyword arguments used specifically to initialize the underlying model. Only used if
model
is a dict. Will override keys in themodel
dict if specified.transform – Transformation to pre-process input time series.
kwargs – Any other keyword arguments (e.g. for initializing a base class). If
model
is a dict, we will also try to pass these arguments when creating the actual underlying model. However, they will not override arguments in either themodel
dict ormodel_kwargs
dict.
- property base_model
The base model at the heart of the full layered model.
- to_dict(_skipped_keys=None)
- Returns
dict with keyword arguments used to initialize the config class.
- classmethod from_dict(config_dict, return_unused_kwargs=False, dim=None, **kwargs)
Constructs a Config from a Python dictionary of parameters.
- Parameters
config_dict (
Dict
[str
,Any
]) – dict that will be used to instantiate this object.return_unused_kwargs – whether to return any unused keyword args.
dim – the dimension of the time series. handled as a special case.
kwargs – any additional parameters to set (overriding config_dict).
- Returns
Config object initialized from the dict.
- get_unused_kwargs(**kwargs)
- class merlion.models.layers.LayeredModel(config=None, model=None, **kwargs)
Bases:
ModelBase
Abstract class implementing a model which wraps around another internal model.
The actual underlying model is stored in
model.config.model
, andmodel.model
is a property which references this. This is to allow the model to retain the initializerLayeredModel(config)
, and to ensure that various attributes do not become de-synchronized (e.g. if we were to storeconfig.model_config
andmodel.model
separately).We define the base model as the non-layered model at the base of the overall model hierarchy.
The layered model is allowed to access any callable attribute of the base model, e.g.
model.set_seasonality(...)
resolves to``model.base_model.set_seasonality(…)`` for a SeasonalityModel. If the base model is a forecaster, the layered model will automatically inherit from ForecasterBase; similarly for DetectorBase or ForecastingDetectorBase. The abstract methods (forecast
andget_anomaly_score
) are overridden to call the underlying model.If the base model is a forecaster, the top-level config
model.config
does not duplicate attributes of the underlying forecaster config (e.g.max_forecast_steps
ortarget_seq_index
). Instead,model.config.max_forecast_steps
will resolve tomodel.config.base_model.max_forecast_steps
. As a result, you will only need to specify this parameter once. The same holds true for DetectorConfig attributes (e.g.threshold
orcalibrator
) when the base model is an anomaly detector.Note
For the time being, every layer of the model is allowed to have its own
transform
. However, after the model is trained, the entire transform will be composed as a single TransformSequence and will be owned by the base model.- config_class
alias of
LayeredModelConfig
- property require_even_sampling: bool
Whether the model assumes that training data is sampled at a fixed frequency
- property require_univariate: bool
Whether the model only works with univariate time series.
- property model
- property base_model
The base model of a base model is itself.
- property train_data
- reset()
Resets the model’s internal state.
- train_pre_process(train_data, **kwargs)
Applies pre-processing steps common for training most models.
- Parameters
train_data (
TimeSeries
) – the original time series of training data- Return type
- Returns
the training data, after any necessary pre-processing has been applied
- train_post_process(train_result, **kwargs)
- class merlion.models.layers.LayeredDetector(config=None, model=None, **kwargs)
Bases:
LayeredModel
,DetectorBase
Base class for a layered anomaly detector. Only to be used as a subclass.
- Parameters
config (
Optional
[LayeredModelConfig
]) – model configuration
- get_anomaly_score(time_series, time_series_prev=None, **kwargs)
Returns the model’s predicted sequence of anomaly scores.
- Parameters
time_series (
TimeSeries
) – the TimeSeries we wish to predict anomaly scores for.time_series_prev (
Optional
[TimeSeries
]) – a TimeSeries immediately precedingtime_series
. If given, we use it to initialize the time series anomaly detection model. Otherwise, we assume thattime_series
immediately follows the training data.
- Return type
- Returns
a univariate TimeSeries of anomaly scores
- class merlion.models.layers.LayeredForecaster(config=None, model=None, **kwargs)
Bases:
LayeredModel
,ForecasterBase
Base class for a layered forecaster. Only to be used as a subclass.
- forecast(time_stamps, time_series_prev=None, **kwargs)
Returns the model’s forecast on the timestamps given. If
self.transform
is specified in the config, the forecast is a forecast of transformed values by default. To invert the transform and forecast the actual values of the time series, specifyinvert_transform = True
when specifying the config.- Parameters
time_stamps – Either a
list
of timestamps we wish to forecast for, or the number of steps (int
) we wish to forecast for.time_series_prev (
Optional
[TimeSeries
]) – a time series immediately precedingtime_series
. If given, we use it to initialize the forecaster’s state. Otherwise, we assume thattime_series
immediately follows the training data.exog_data – A time series of exogenous variables. Exogenous variables are known a priori, and they are independent of the variable being forecasted.
exog_data
must include data for all oftime_stamps
; iftime_series_prev
is given, it must include data for all oftime_series_prev.time_stamps
as well. Optional. Only supported for models which inherit from ForecasterExogBase.return_iqr – whether to return the inter-quartile range for the forecast. Only supported for models which return error bars.
return_prev – whether to return the forecast for
time_series_prev
(and its stderr or IQR if relevant), in addition to the forecast fortime_stamps
. Only used iftime_series_prev
is provided.
- Returns
(forecast, stderr)
ifreturn_iqr
is false,(forecast, lb, ub)
otherwise.forecast
: the forecast for the timestamps givenstderr
: the standard error of each forecast value. May beNone
.lb
: 25th percentile of forecast values for each timestampub
: 75th percentile of forecast values for each timestamp
- class merlion.models.layers.LayeredForecastingDetector(config=None, model=None, **kwargs)
Bases:
LayeredForecaster
,LayeredDetector
,ForecastingDetectorBase
Base class for a layered forecasting detector. Only to be used as a subclass.
- Parameters
config (
Optional
[LayeredModelConfig
]) – model configuration