mlvern.core package

Submodules

mlvern.core.forge module

class mlvern.core.forge.Forge(project: str, base_dir: str = '.')

Bases: object

evaluate(run_id_or_model: Any, X_test, y_test, output_dir: str | None = None) → Dict[str, Any]

Evaluate a model and generate evaluation metrics and plots.

Args:

run_id_or_model: Either a run_id (str) or a model object X_test: Test features y_test: Test labels output_dir: Directory to save evaluation plots

Returns:

Dict with metrics and paths to generated plots

get_dataset_path(dataset_hash: str) → str

Get the filesystem path for a dataset by hash.

Args:

dataset_hash: The dataset hash identifier

Returns:

Absolute path to the dataset directory

Raises:

ValueError: If dataset hash not found

get_dataset_report(dataset_hash: str) → Dict[str, Any]

Get aggregated report for a dataset.

Includes inspection, statistics, risk, and EDA reports.

Args:

dataset_hash: Hash of the dataset

Returns:

Aggregated report dict

get_project_stats() → Dict[str, Any]

Get overall project statistics.

Returns:

Dictionary with dataset count, run count, total size, etc.

get_run(run_id: str) → Dict[str, Any]

Get run metadata and information by run_id.

Args:

run_id: The run identifier

Returns:

Dictionary with run metadata, metrics, and paths

Raises:

ValueError: If run not found

get_run_artifacts(run_id: str) → Dict[str, str]

Get paths to all artifacts for a run (model, config, metrics, etc).

Args:

run_id: The run identifier

Returns:

Dictionary mapping artifact names to their filesystem paths

get_run_metrics(run_id: str) → Dict[str, Any]

Get metrics for a specific run.

Args:

run_id: The run identifier

Returns:

Metrics dictionary

get_run_tags(run_id: str) → Dict[str, Any]

Get tags for a specific run.

Args:

run_id: The run identifier

Returns:

Dictionary of tags

init()

Initialize the mlvern project directory structure.

list_datasets()

List all registered datasets in the project.

list_models() → Dict[str, Any]

List all registered models.

list_runs() → Dict[str, Any]

List all runs in the project.

load_dataset(dataset_hash: str) → Dict[str, Any] | None

Load dataset metadata and paths by hash.

Args:

dataset_hash: The dataset hash identifier

Returns:

Dictionary containing dataset info and paths to reports/plots

Raises:

ValueError: If dataset not found

load_dataset_by_hash(dataset_hash: str) → DataFrame

Load a dataset from storage by hash.

Args:

dataset_hash: Hash of the dataset

Returns:

Loaded DataFrame

Raises:

FileNotFoundError: If dataset not found

load_model(run_id: str, safe: bool = True) → Any

Load a trained model from a run.

Args:

run_id: The run identifier safe: If True, warn about pickle security risks

Returns:

The loaded model object

Raises:

ValueError: If run not found or model not found

predict(run_id_or_model: Any, X_test) → Any

Make predictions using a model from a run or passed model object.

Args:

run_id_or_model: Either a run_id (str) or a model object X_test: Test data for predictions

Returns:

Predictions array

prune_datasets(older_than_days: int = 30, confirm: bool = False) → List[str]

Remove datasets older than specified number of days.

Args:

older_than_days: Remove datasets older than this many days confirm: Must be True to perform deletion (safety check)

Returns:

List of removed dataset hashes

register_dataset(df, target: str)

register_model(model: Any, metadata: Dict[str, Any], model_id: str | None = None) → str

Register a model in the model registry.

Args:

model: The model object (will be saved) metadata: Metadata dict (should include: model_name, source_run_id, description, hyperparameters, etc.) model_id: Optional custom model ID; auto-generated if not provided

Returns:

The model ID

remove_run(run_id: str, confirm: bool = False) → bool

Remove a run and its artifacts.

Args:

run_id: The run identifier confirm: Must be True to perform deletion (safety check)

Returns:

True if removal succeeded, False otherwise

run(model, X_train, y_train, X_val, y_val, config: dict, dataset_fp)

Train a model and create a run record.

save_dataset(df: DataFrame, dataset_hash: str, name: str | None = None, tags: Dict[str, Any] | None = None) → Dict[str, Any]

Save a dataframe to an existing dataset directory.

Args:

df: DataFrame to save dataset_hash: Hash of the dataset name: Optional friendly name for the dataset tags: Optional tags dict

Returns:

Dict with save info

tag_run(run_id: str, tags: Dict[str, Any]) → None

Add or update tags on a run for searchability.

Args:

run_id: The run identifier tags: Dictionary of tags to add/update

Module contents

class mlvern.core.Forge(project: str, base_dir: str = '.')

Bases: object

evaluate(run_id_or_model: Any, X_test, y_test, output_dir: str | None = None) → Dict[str, Any]

Evaluate a model and generate evaluation metrics and plots.

Args:

run_id_or_model: Either a run_id (str) or a model object X_test: Test features y_test: Test labels output_dir: Directory to save evaluation plots

Returns:

Dict with metrics and paths to generated plots

get_dataset_path(dataset_hash: str) → str

Get the filesystem path for a dataset by hash.

Args:

dataset_hash: The dataset hash identifier

Returns:

Absolute path to the dataset directory

Raises:

ValueError: If dataset hash not found

get_dataset_report(dataset_hash: str) → Dict[str, Any]

Get aggregated report for a dataset.

Includes inspection, statistics, risk, and EDA reports.

Args:

dataset_hash: Hash of the dataset

Returns:

Aggregated report dict

get_project_stats() → Dict[str, Any]

Get overall project statistics.

Returns:

Dictionary with dataset count, run count, total size, etc.

get_run(run_id: str) → Dict[str, Any]

Get run metadata and information by run_id.

Args:

run_id: The run identifier

Returns:

Dictionary with run metadata, metrics, and paths

Raises:

ValueError: If run not found

get_run_artifacts(run_id: str) → Dict[str, str]

Get paths to all artifacts for a run (model, config, metrics, etc).

Args:

run_id: The run identifier

Returns:

Dictionary mapping artifact names to their filesystem paths

get_run_metrics(run_id: str) → Dict[str, Any]

Get metrics for a specific run.

Args:

run_id: The run identifier

Returns:

Metrics dictionary

get_run_tags(run_id: str) → Dict[str, Any]

Get tags for a specific run.

Args:

run_id: The run identifier

Returns:

Dictionary of tags

init()

Initialize the mlvern project directory structure.

list_datasets()

List all registered datasets in the project.

list_models() → Dict[str, Any]

List all registered models.

list_runs() → Dict[str, Any]

List all runs in the project.

load_dataset(dataset_hash: str) → Dict[str, Any] | None

Load dataset metadata and paths by hash.

Args:

dataset_hash: The dataset hash identifier

Returns:

Dictionary containing dataset info and paths to reports/plots

Raises:

ValueError: If dataset not found

load_dataset_by_hash(dataset_hash: str) → DataFrame

Load a dataset from storage by hash.

Args:

dataset_hash: Hash of the dataset

Returns:

Loaded DataFrame

Raises:

FileNotFoundError: If dataset not found

load_model(run_id: str, safe: bool = True) → Any

Load a trained model from a run.

Args:

run_id: The run identifier safe: If True, warn about pickle security risks

Returns:

The loaded model object

Raises:

ValueError: If run not found or model not found

predict(run_id_or_model: Any, X_test) → Any

Make predictions using a model from a run or passed model object.

Args:

run_id_or_model: Either a run_id (str) or a model object X_test: Test data for predictions

Returns:

Predictions array

prune_datasets(older_than_days: int = 30, confirm: bool = False) → List[str]

Remove datasets older than specified number of days.

Args:

older_than_days: Remove datasets older than this many days confirm: Must be True to perform deletion (safety check)

Returns:

List of removed dataset hashes

register_dataset(df, target: str)

register_model(model: Any, metadata: Dict[str, Any], model_id: str | None = None) → str

Register a model in the model registry.

Args:

model: The model object (will be saved) metadata: Metadata dict (should include: model_name, source_run_id, description, hyperparameters, etc.) model_id: Optional custom model ID; auto-generated if not provided

Returns:

The model ID

remove_run(run_id: str, confirm: bool = False) → bool

Remove a run and its artifacts.

Args:

run_id: The run identifier confirm: Must be True to perform deletion (safety check)

Returns:

True if removal succeeded, False otherwise

run(model, X_train, y_train, X_val, y_val, config: dict, dataset_fp)

Train a model and create a run record.

save_dataset(df: DataFrame, dataset_hash: str, name: str | None = None, tags: Dict[str, Any] | None = None) → Dict[str, Any]

Save a dataframe to an existing dataset directory.

Args:

df: DataFrame to save dataset_hash: Hash of the dataset name: Optional friendly name for the dataset tags: Optional tags dict

Returns:

Dict with save info

tag_run(run_id: str, tags: Dict[str, Any]) → None

Add or update tags on a run for searchability.

Args:

run_id: The run identifier tags: Dictionary of tags to add/update