trphysx.config¶

trphysx.config.arg_parser¶

class trphysx.config.arg_parser.DataClass(*args, **kwargs)¶: Bases: typing.Protocol

class trphysx.config.arg_parser.HfArgumentParser(dataclass_types: Union[trphysx.config.arg_parser.DataClass, Iterable[trphysx.config.arg_parser.DataClass]], **kwargs)¶

Bases: argparse.ArgumentParser

This subclass of argparse.ArgumentParser uses type hints on dataclasses to generate arguments. The class is designed to play well with the native argparse. In particular, you can add more (non-dataclass backed) arguments to the parser after initialization and you’ll get the output back after parsing as an additional namespace.

Code originally from the Huggingface Transformers repository: https://github.com/huggingface/transformers/blob/master/src/transformers/hf_argparser.py

Parameters:	dataclass_types (Union[DataClass, Iterable[DataClass]]) – Dataclass type, or list of dataclass types for which we will “fill” instances with the parsed args. kwargs (optional) – Passed to argparse.ArgumentParser() in the regular way.

parse_args_into_dataclasses(args: Iterable[str] = None, return_remaining_strings: bool = False, look_for_args_file: bool = True, args_filename: str = None) → Tuple[trphysx.config.arg_parser.DataClass]¶

Parse command-line args into instances of the specified dataclass types.

Parameters:

args (Iterable[str]) – List of strings to parse. The default is taken from sys.argv. (same as argparse.ArgumentParser)
return_remaining_strings (bool) – If true, also return a list of remaining argument strings.
look_for_args_file (bool) – If true, will look for a “.args” file with the same base name as the entry point script for this process, and will append its potential content to the command line args.
args_filename (str) – If not None, will uses this file instead of the “.args” file specified in the previous argument.

Returns:

the dataclass instances in the same order as they were passed to the initializer.abspath
if applicable, an additional namespace for more (non-dataclass backed) arguments added to the parser after initialization.
The potential list of remaining argument strings. (same as argparse.ArgumentParser.parse_known_args)

Return type:

(Tuple[DataClass])

trphysx.config.args¶

class trphysx.config.args.ModelArguments(init_name: str = 'lorenz', model_name: str = None, config_name: str = None, embedding_name: str = None, embedding_file_or_path: str = None, transformer_file_or_path: str = None, viz_name: str = None)¶

Bases: object

Arguments pertaining to which model/config/tokenizer we are going to fine-tune, or train from scratch.

init_name = 'lorenz'¶

model_name = None¶

config_name = None¶

embedding_name = None¶

embedding_file_or_path = None¶

transformer_file_or_path = None¶

viz_name = None¶

class trphysx.config.args.DataArguments(n_train: int = 2048, n_eval: int = 256, stride: int = 32, training_h5_file: str = None, eval_h5_file: str = None, overwrite_cache: bool = False, cache_path: str = None)¶

Bases: object

Arguments pertaining to training and evaluation data.

n_train = 2048¶

n_eval = 256¶

stride = 32¶

training_h5_file = None¶

eval_h5_file = None¶

overwrite_cache = False¶

cache_path = None¶

class trphysx.config.args.TrainingArguments(block_size: int = -1, exp_dir: str = None, ckpt_dir: str = None, plot_dir: str = None, save_steps: int = 25, eval_steps: int = 25, plot_max: int = 3, epoch_start: int = 0, epochs: int = 200, lr: float = 0.001, max_grad_norm: float = 0.1, dataloader_drop_last: bool = True, gradient_accumulation_steps: int = 1, train_batch_size: int = 256, eval_batch_size: int = 16, local_rank: int = -1, n_gpu: int = 1, seed: int = 12345, notes: str = None)¶

Bases: object

Arguments pertaining to what data we are going to input our model for training and eval.

block_size = -1¶

exp_dir = None¶

ckpt_dir = None¶

plot_dir = None¶

save_steps = 25¶

eval_steps = 25¶

plot_max = 3¶

epoch_start = 0¶

epochs = 200¶

lr = 0.001¶

max_grad_norm = 0.1¶

dataloader_drop_last = True¶

gradient_accumulation_steps = 1¶

train_batch_size = 256¶

eval_batch_size = 16¶

local_rank = -1¶

n_gpu = 1¶

seed = 12345¶

notes = None¶

class trphysx.config.args.ArgUtils¶

Bases: object

Argument utility class for modifying particular arguments after initialization

classmethod config(modelArgs: trphysx.config.args.ModelArguments, dataArgs: trphysx.config.args.DataArguments, trainingArgs: trphysx.config.args.TrainingArguments, create_paths: bool = True) → Tuple[trphysx.config.args.ModelArguments, trphysx.config.args.DataArguments, trphysx.config.args.TrainingArguments]¶

Runs additional runtime configuration updates for argument instances

Parameters:	modelArgs (ModelArguments) – Transformer model arguments dataArgs (DataArguments) – Data loader/ data set arguments trainingArgs (TrainingArguments) – Training arguments create_paths (bool, optional) – Create training/testing folders. Defaults to True.
Returns:	Updated argument instances
Return type:	Tuple[ModelArguments, DataArguments, TrainingArguments]

classmethod configModelNames(modelArgs: trphysx.config.args.ModelArguments) → trphysx.config.args.ModelArguments¶

classmethod configPaths(modelArgs: trphysx.config.args.ModelArguments, dataArgs: trphysx.config.args.DataArguments, trainingArgs: trphysx.config.args.TrainingArguments) → Tuple[trphysx.config.args.ModelArguments, trphysx.config.args.DataArguments, trphysx.config.args.TrainingArguments]¶

Sets up various folder path parameters

Parameters:	modelArgs (ModelArguments) – Transformer model arguments dataArgs (DataArguments) – Data loader/ data set arguments trainingArgs (TrainingArguments) – Training arguments
Returns:	Updated argument instances
Return type:	Tuple[ModelArguments, DataArguments, TrainingArguments]

classmethod configTorchDevices(args: trphysx.config.args.TrainingArguments) → trphysx.config.args.TrainingArguments¶

Sets up device ids for training

Parameters:	args (TrainingArguments) – Training arguments
Returns:	Updated argument instance
Return type:	TrainingArguments

trphysx.config.configuration_auto¶

class trphysx.config.configuration_auto.AutoPhysConfig¶

Bases: object

Helper class for creating configurations for different built in examples

Raises:	EnvironmentError – If direct initialization of this class is attempted.

classmethod load_config(model_name_or_path, **kwargs) → trphysx.config.configuration_phys.PhysConfig¶

Creates a configuration object for a transformer model. Predefined configs currently support: “lorenz”, “cylinder”, “grayscott”

Parameters:	model_name_or_path (str) – Name of model or path to save config JSON file
Returns:	Configuration of transformer
Return type:	(PhysConfig)

classmethod from_json_file(json_file: str) → Dict[KT, VT]¶

Reads a json file and loads it into a dictionary.

Parameters:	json_file (string) – Path to the JSON file containing the parameters.
Returns:	Dictionary of parsed JSON
Return type:	Dict

trphysx.config.configuration_cylinder¶

class trphysx.config.configuration_cylinder.CylinderConfig(n_ctx=16, n_embd=128, n_layer=6, n_head=4, state_dims=[3, 64, 128], activation_function='gelu_new', **kwargs)¶

Bases: trphysx.config.configuration_phys.PhysConfig

This is the configuration class for the modeling of the flow around a cylinder system.

model_type = 'cylinder'¶

hidden_size¶

num_attention_heads¶

num_hidden_layers¶

trphysx.config.configuration_grayscott¶

class trphysx.config.configuration_grayscott.GrayScottConfig(n_ctx=128, n_embd=512, n_layer=2, n_head=32, state_dims=[2, 32, 32, 32], activation_function='gelu_new', **kwargs)¶

Bases: trphysx.config.configuration_phys.PhysConfig

This is the configuration class for the modeling of the Gray-scott system.

model_type = 'cylinder'¶

hidden_size¶

num_attention_heads¶

num_hidden_layers¶

trphysx.config.configuration_lorenz¶

class trphysx.config.configuration_lorenz.LorenzConfig(n_ctx=64, n_embd=32, n_layer=4, n_head=4, state_dims=[3], activation_function='gelu_new', initializer_range=0.05, **kwargs)¶

Bases: trphysx.config.configuration_phys.PhysConfig

This is the configuration class for the modeling of the Lorenz system.

model_type = 'lorenz'¶

hidden_size¶

num_attention_heads¶

num_hidden_layers¶

trphysx.config.configuration_phys¶

class trphysx.config.configuration_phys.PhysConfig(**kwargs)¶

Bases: object

Parent class for physical transformer configuration. This is a slimmed version of the pretrainedconfig from the Hugging Face repository.

Parameters:

n_ctx (int) – Context window of transformer model.
n_embd (int) – Dimensionality of the embeddings and hidden states.
n_layer (int) – Number of hidden layers in the transformer.
n_head (int) – Number of self-attention heads in each layer.
state_dims (List) – List of physical state dimensionality. Used in embedding models.
activation_function (str, optional) – Activation function. Defaults to “gelu_new”.
resid_pdrop (float, optional) – The dropout probability for all fully connected layers in the transformer. Defaults to 0.0.
embd_pdrop (float, optional) – The dropout ratio for the embeddings. Defaults to 0.0.
attn_pdrop (float, optional) – The dropout ratio for the multi-head attention. Defaults to 0.0.
layer_norm_epsilon (float, optional) – The epsilon to use in the layer normalization layers. Defaults to 1e-5.
initializer_range (float, optional) – The standard deviation for initializing all weight matrices. Defaults to 0.02.
output_hidden_states (bool, optional) – Output embeddeding states from transformer. Defaults to False.
output_attentions (bool, optional) – Output attention values from transformer. Defaults to False.
use_cache (bool, optional) – Store transformers internal state for rapid predictions. Defaults to True.

Raises:

AssertionError – If provided parameter is not a config parameter

model_type = ''¶

save_pretrained(save_directory: str) → None¶

Save a configuration object to JSON file.

Parameters:	save_directory (str) – Directory where the configuration JSON file will be saved.
Raises:	AssertionError – If provided directory does not exist.

classmethod from_dict(config_dict: Dict[str, any], **kwargs) → trphysx.config.configuration_phys.PhysConfig¶

Constructs a config from a Python dictionary of parameters.

Parameters:	config_dict (Dict[str, any]) – Dictionary of parameters. kwargs (Dict[str, any]) – Additional parameters from which to initialize the configuration object.
Returns:	An instance of a configuration object
Return type:	(PhysConfig)

to_dict() → Dict[str, any]¶

Serializes this instance to a Python dictionary.

Returns:	Dictionary of config attributes
Return type:	(Dict[str, any])

to_json_string() → str¶

Serializes this instance to a JSON string.

Returns:	String of configuration instance in JSON format.
Return type:	(str)

to_json_file(json_file_path: str) → None¶

Save config instance to JSON file.

Parameters:	json_file_path (str) – Path to the JSON file in which this configuration instance’s parameters will be saved.

update(config_dict: Dict[KT, VT]) → None¶

Updates attributes of this class with attributes from provided dictionary.

Parameters:	config_dict (Dict) – Dictionary of attributes that shall be updated for this class.