trphysx.config¶
trphysx.config.arg_parser¶
-
class
trphysx.config.arg_parser.
DataClass
(*args, **kwargs)¶ Bases:
typing.Protocol
-
class
trphysx.config.arg_parser.
HfArgumentParser
(dataclass_types: Union[trphysx.config.arg_parser.DataClass, Iterable[trphysx.config.arg_parser.DataClass]], **kwargs)¶ Bases:
argparse.ArgumentParser
This subclass of argparse.ArgumentParser uses type hints on dataclasses to generate arguments. The class is designed to play well with the native argparse. In particular, you can add more (non-dataclass backed) arguments to the parser after initialization and you’ll get the output back after parsing as an additional namespace.
Code originally from the Huggingface Transformers repository: https://github.com/huggingface/transformers/blob/master/src/transformers/hf_argparser.py
Parameters: -
parse_args_into_dataclasses
(args: Iterable[str] = None, return_remaining_strings: bool = False, look_for_args_file: bool = True, args_filename: str = None) → Tuple[trphysx.config.arg_parser.DataClass]¶ Parse command-line args into instances of the specified dataclass types.
Parameters: - args (Iterable[str]) – List of strings to parse. The default is taken from sys.argv. (same as argparse.ArgumentParser)
- return_remaining_strings (bool) – If true, also return a list of remaining argument strings.
- look_for_args_file (bool) – If true, will look for a “.args” file with the same base name as the entry point script for this process, and will append its potential content to the command line args.
- args_filename (str) – If not None, will uses this file instead of the “.args” file specified in the previous argument.
Returns: - the dataclass instances in the same order as they were passed to the initializer.abspath
- if applicable, an additional namespace for more (non-dataclass backed) arguments added to the parser after initialization.
- The potential list of remaining argument strings. (same as argparse.ArgumentParser.parse_known_args)
Return type: (Tuple[DataClass])
-
trphysx.config.args¶
-
class
trphysx.config.args.
ModelArguments
(init_name: str = 'lorenz', model_name: str = None, config_name: str = None, embedding_name: str = None, embedding_file_or_path: str = None, transformer_file_or_path: str = None, viz_name: str = None)¶ Bases:
object
Arguments pertaining to which model/config/tokenizer we are going to fine-tune, or train from scratch.
-
init_name
= 'lorenz'¶
-
model_name
= None¶
-
config_name
= None¶
-
embedding_name
= None¶
-
embedding_file_or_path
= None¶
-
transformer_file_or_path
= None¶
-
viz_name
= None¶
-
-
class
trphysx.config.args.
DataArguments
(n_train: int = 2048, n_eval: int = 256, stride: int = 32, training_h5_file: str = None, eval_h5_file: str = None, overwrite_cache: bool = False, cache_path: str = None)¶ Bases:
object
Arguments pertaining to training and evaluation data.
-
n_train
= 2048¶
-
n_eval
= 256¶
-
stride
= 32¶
-
training_h5_file
= None¶
-
eval_h5_file
= None¶
-
overwrite_cache
= False¶
-
cache_path
= None¶
-
-
class
trphysx.config.args.
TrainingArguments
(block_size: int = -1, exp_dir: str = None, ckpt_dir: str = None, plot_dir: str = None, save_steps: int = 25, eval_steps: int = 25, plot_max: int = 3, epoch_start: int = 0, epochs: int = 200, lr: float = 0.001, max_grad_norm: float = 0.1, dataloader_drop_last: bool = True, gradient_accumulation_steps: int = 1, train_batch_size: int = 256, eval_batch_size: int = 16, local_rank: int = -1, n_gpu: int = 1, seed: int = 12345, notes: str = None)¶ Bases:
object
Arguments pertaining to what data we are going to input our model for training and eval.
-
block_size
= -1¶
-
exp_dir
= None¶
-
ckpt_dir
= None¶
-
plot_dir
= None¶
-
save_steps
= 25¶
-
eval_steps
= 25¶
-
plot_max
= 3¶
-
epoch_start
= 0¶
-
epochs
= 200¶
-
lr
= 0.001¶
-
max_grad_norm
= 0.1¶
-
dataloader_drop_last
= True¶
-
gradient_accumulation_steps
= 1¶
-
train_batch_size
= 256¶
-
eval_batch_size
= 16¶
-
local_rank
= -1¶
-
n_gpu
= 1¶
-
seed
= 12345¶
-
notes
= None¶
-
-
class
trphysx.config.args.
ArgUtils
¶ Bases:
object
Argument utility class for modifying particular arguments after initialization
-
classmethod
config
(modelArgs: trphysx.config.args.ModelArguments, dataArgs: trphysx.config.args.DataArguments, trainingArgs: trphysx.config.args.TrainingArguments, create_paths: bool = True) → Tuple[trphysx.config.args.ModelArguments, trphysx.config.args.DataArguments, trphysx.config.args.TrainingArguments]¶ Runs additional runtime configuration updates for argument instances
Parameters: - modelArgs (ModelArguments) – Transformer model arguments
- dataArgs (DataArguments) – Data loader/ data set arguments
- trainingArgs (TrainingArguments) – Training arguments
- create_paths (bool, optional) – Create training/testing folders. Defaults to True.
Returns: Updated argument instances
Return type: Tuple[ModelArguments, DataArguments, TrainingArguments]
-
classmethod
configModelNames
(modelArgs: trphysx.config.args.ModelArguments) → trphysx.config.args.ModelArguments¶
-
classmethod
configPaths
(modelArgs: trphysx.config.args.ModelArguments, dataArgs: trphysx.config.args.DataArguments, trainingArgs: trphysx.config.args.TrainingArguments) → Tuple[trphysx.config.args.ModelArguments, trphysx.config.args.DataArguments, trphysx.config.args.TrainingArguments]¶ Sets up various folder path parameters
Parameters: - modelArgs (ModelArguments) – Transformer model arguments
- dataArgs (DataArguments) – Data loader/ data set arguments
- trainingArgs (TrainingArguments) – Training arguments
Returns: Updated argument instances
Return type: Tuple[ModelArguments, DataArguments, TrainingArguments]
-
classmethod
configTorchDevices
(args: trphysx.config.args.TrainingArguments) → trphysx.config.args.TrainingArguments¶ Sets up device ids for training
Parameters: args (TrainingArguments) – Training arguments Returns: Updated argument instance Return type: TrainingArguments
-
classmethod
trphysx.config.configuration_auto¶
-
class
trphysx.config.configuration_auto.
AutoPhysConfig
¶ Bases:
object
Helper class for creating configurations for different built in examples
Raises: EnvironmentError – If direct initialization of this class is attempted. -
classmethod
load_config
(model_name_or_path, **kwargs) → trphysx.config.configuration_phys.PhysConfig¶ Creates a configuration object for a transformer model. Predefined configs currently support: “lorenz”, “cylinder”, “grayscott”
Parameters: model_name_or_path (str) – Name of model or path to save config JSON file Returns: Configuration of transformer Return type: (PhysConfig)
-
classmethod
from_json_file
(json_file: str) → Dict[KT, VT]¶ Reads a json file and loads it into a dictionary.
Parameters: json_file (string) – Path to the JSON file containing the parameters. Returns: Dictionary of parsed JSON Return type: Dict
-
classmethod
trphysx.config.configuration_cylinder¶
-
class
trphysx.config.configuration_cylinder.
CylinderConfig
(n_ctx=16, n_embd=128, n_layer=6, n_head=4, state_dims=[3, 64, 128], activation_function='gelu_new', **kwargs)¶ Bases:
trphysx.config.configuration_phys.PhysConfig
This is the configuration class for the modeling of the flow around a cylinder system.
-
model_type
= 'cylinder'¶
-
num_attention_heads
¶
-
trphysx.config.configuration_grayscott¶
-
class
trphysx.config.configuration_grayscott.
GrayScottConfig
(n_ctx=128, n_embd=512, n_layer=2, n_head=32, state_dims=[2, 32, 32, 32], activation_function='gelu_new', **kwargs)¶ Bases:
trphysx.config.configuration_phys.PhysConfig
This is the configuration class for the modeling of the Gray-scott system.
-
model_type
= 'cylinder'¶
-
num_attention_heads
¶
-
trphysx.config.configuration_lorenz¶
-
class
trphysx.config.configuration_lorenz.
LorenzConfig
(n_ctx=64, n_embd=32, n_layer=4, n_head=4, state_dims=[3], activation_function='gelu_new', initializer_range=0.05, **kwargs)¶ Bases:
trphysx.config.configuration_phys.PhysConfig
This is the configuration class for the modeling of the Lorenz system.
-
model_type
= 'lorenz'¶
-
num_attention_heads
¶
-
trphysx.config.configuration_phys¶
-
class
trphysx.config.configuration_phys.
PhysConfig
(**kwargs)¶ Bases:
object
Parent class for physical transformer configuration. This is a slimmed version of the pretrainedconfig from the Hugging Face repository.
Parameters: - n_ctx (int) – Context window of transformer model.
- n_embd (int) – Dimensionality of the embeddings and hidden states.
- n_layer (int) – Number of hidden layers in the transformer.
- n_head (int) – Number of self-attention heads in each layer.
- state_dims (List) – List of physical state dimensionality. Used in embedding models.
- activation_function (str, optional) – Activation function. Defaults to “gelu_new”.
- resid_pdrop (float, optional) – The dropout probability for all fully connected layers in the transformer. Defaults to 0.0.
- embd_pdrop (float, optional) – The dropout ratio for the embeddings. Defaults to 0.0.
- attn_pdrop (float, optional) – The dropout ratio for the multi-head attention. Defaults to 0.0.
- layer_norm_epsilon (float, optional) – The epsilon to use in the layer normalization layers. Defaults to 1e-5.
- initializer_range (float, optional) – The standard deviation for initializing all weight matrices. Defaults to 0.02.
- output_hidden_states (bool, optional) – Output embeddeding states from transformer. Defaults to False.
- output_attentions (bool, optional) – Output attention values from transformer. Defaults to False.
- use_cache (bool, optional) – Store transformers internal state for rapid predictions. Defaults to True.
Raises: AssertionError – If provided parameter is not a config parameter
-
model_type
= ''¶
-
save_pretrained
(save_directory: str) → None¶ Save a configuration object to JSON file.
Parameters: save_directory (str) – Directory where the configuration JSON file will be saved. Raises: AssertionError – If provided directory does not exist.
-
classmethod
from_dict
(config_dict: Dict[str, any], **kwargs) → trphysx.config.configuration_phys.PhysConfig¶ Constructs a config from a Python dictionary of parameters.
Parameters: - config_dict (Dict[str, any]) – Dictionary of parameters.
- kwargs (Dict[str, any]) – Additional parameters from which to initialize the configuration object.
Returns: An instance of a configuration object
Return type:
-
to_dict
() → Dict[str, any]¶ Serializes this instance to a Python dictionary.
Returns: Dictionary of config attributes Return type: (Dict[str, any])
-
to_json_string
() → str¶ Serializes this instance to a JSON string.
Returns: String of configuration instance in JSON format. Return type: (str)
-
to_json_file
(json_file_path: str) → None¶ Save config instance to JSON file.
Parameters: json_file_path (str) – Path to the JSON file in which this configuration instance’s parameters will be saved.
-
update
(config_dict: Dict[KT, VT]) → None¶ Updates attributes of this class with attributes from provided dictionary.
Parameters: config_dict (Dict) – Dictionary of attributes that shall be updated for this class.