A specific type of planned Constraint that describes a constraint which can be evaluated automatically against corresponding accountability trace. Such constraint would normally link to its implementation (e.g., a SHACL rule) using the https://w3id.org/ep-plan#hasConstraintImplementation property.
A sao:InformationElement which records information detailing a piece of data, software or hardware that may be realized during the Implementation stage. A reused object (e.g., public training datasets) is also considered as :RealizableObject during the Design stage as it is not yet implemented (i.e., physical copies have not yet been integrated withing the ML pipeline)
A relation that links some evaluation measure to one or more results produced for this measure.
A specific type of planned sao:AccountableResult which represents the characteristics, performance, etc. related to all the implemented hardware and software 'plumbing' that is not generally considered part of the AI processes, however, may also be source of erroneous behaviour in AI systems. It is advisable that the information realization corresponding to the SupportingInfrastructureComponent variable contains the following information elements: 1. SupportingInfrastructure, the realized supporting infrastructure should be an instance of RealizedObject. If the implementation deviates from the design, then an additional upporting infrastructure should be indicated as an instance of ExcludedObject. 2. Limitation 3. Risk 4. Bias 5. Tradeoff 6. SystemAuditMechanism 7. SystemExplanationMechanims 8. SystemHumanOversightMechanism 9. SystemSecurityMechanism 10. HardLaw 11. SoftLaw
A specific type of design specification which represents the description of planned requirements, characteristics, risks, etc. related to a dataset (e.g., a training dataset for ML model) expected to be used as part of the AI system life cycle. It is advisable that the information realization corresponding to DatasetSpecification contains the following information elements 1. Dataset 2. DatasetCharacteristic 3. DataCollectionProcedure 4. DataPreprocesingProcedure 5. IntendedUseCase 6. Limitation 7. Risk 8. Bias 9. Tradeoff 10. IntendedUserGroup 11. IncorrectUseCase 12. HardLaw 13. SoftLaw It is discouraged that the information realization corresponding to the Dataset Specification contains the following information elements: 1. SystemAuditMechanism 2. SystemExplanationMechanims 3. SystemHumanOversightMechanism 4. SystemSecurityMechanism 5. SystemIntegrationGuidance 6. SystemDeploymentGuidance 7. SystemDocumentationGuidance 8. SystemIntegrationGuidance 9. SystemUsageGuidance 10. Model 11. ModelCharacteristic 12. Algorithm
A specific type of planned sao:AccountableResult which represents the description of a realized ML Model, including it characteristics, performance, etc. It is advisable that the information realization corresponding to the ModelComponent variable contains the following information elements: 1. Model: the realized model should be an instance of RealizedObject. If the implementation deviates from the design, then an additional model should be indicated as an instance of ExcludedObject. 2. ModelCharacteristic 3. Algorithm 4. Parameter 5. IntendedUseCase 6. Limitation 7. Risk 8. Bias 9. Tradeoff 10. IncorrectUseCase 11. IntendedUserGroup 12. HardLaw 13. SoftLaw It is discouraged that the information realization corresponding to the ModelComponent variable contains the following information elements: 1. SystemAuditMechanism 2. SystemExplanationMechanims 3. SystemHumanOversightMechanism 4. SystemSecurityMechanism 5. SystemIntegrationGuidance 6. SystemDeploymentGuidance 7. SystemDocumentationGuidance 8. SystemIntegrationGuidance 9. SystemUsageGuidance
A legal document giving official permission to do something with a resource.
A specific type of a planned human compliance assurance decision representing certification granting. This is usually done by third parties, but may also be done inhouse to prove that the AI System abides by internal policies.
A sao:InformnationElement which records information describing the justification behind a deviation from the Design Specification. For example, the reason behind realizing a dataset other than the one indicated by the Dataset Design Specification was used, or why an evaluation measure was recorded which differed from the one indicated to be expected in an Evaluation Specification.
A value describing the format of an output. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A specific type of :Guidance which records a specific piece of information about the guidance to how the :AI_System should be maintained.
A specific type of planned sao:AccountableResult which represents a high level reference to a guideline (e.g. a guideline document) that is expected to be produced during the implementation in order to inform on the correct procedure for deploying, using and maintaining the system or any of its components.
A planned step describing the creation of a guideline that has some impact on the AI System, e.g. the production of the deployment guideline describing how the AI System is to be deployed on-site or the production of the maintenance guideline describing how the AI System is to be maintained.
A specific type of :Guidance which records a specific piece of information about the guidance to how the :AI_System should be integrated with other systems or services.
A planned step describing the creation of a specification that has some impact on the AI System, e.g. the production of the design specification describing the training dataset to be used. This step is typically part of the Design stage of an AI System's life cycle.
A planned step describing an evaluation, i.e, testing and producing evaluation metrics, that has some impact on the AI System, e.g. the evaluation of the ML model using a specified Dataset or the :AI_System undergoing a third party evaluation.
A type of :RealizableObjectCharacteristic detailing how :AI_System may be interrogated about its inner processes and outputs produced.
A planned step describing the realization of a component. This includes implementing a component or retrieving it from a different (or third party) source. The realization of the component has some impact on the AI System, e.g. the implementation of the ML model or importing a third party dataset.
A sao:InformationElement which records a specific piece of information describing a model's parameter. This is broader than mls:HyperParameter which describes only a hyperparameter of a model.
A specific type of design specification which represents the description of planned requirements, characteristics, performance, etc. related to an AI model. It is advisable that the information realization corresponding to ModelSpecification contains the following information elements: 1. Model 2. ModelCharacteristic 3. Algorithm 4. IntendedUseCase 5. Limitation 6. Risk 7. Bias 8. Tradeoff 9. IntendedUserGroup 10. Parameter 11. Hyperparameter 12. IncorrectUseCase 13. HardLaw 14. SoftLaw It is discouraged that the information realization corresponding to the ModelSpecification contains the following information elements: 1. SystemAuditMechanism 2. SystemExplanationMechanims 3. SystemHumanOversightMechanism 4. SystemSecurityMechanism 5. SystemIntegrationGuidance 6. SystemDeploymentGuidance 7. SystemDocumentationGuidance 8. SystemIntegrationGuidance 9. SystemUsageGuidance
A specific type of :Guideline which represents the description of guidelines for the Operation stage of the AI System. It is advisable that the information realization corresponding to the System Operation Guideline variable contains the following information elements: Guidance. 1. SystemIntegrationGuidance 2. SystemUsageGuidance
A specific type of planned sao:AccountableResult which represents the description of an realized dataset, including its characteristics, how it was collected, its risks, etc. The dataset component represents a collection of collected and pre-processed data. Once it is split, the subclasses TestingDataset, ValidationDataset, and EvaluationDataset can be used. It is advisable that the information realization corresponding to the DatasetComponent variable contains the following information elements: 1. Dataset, the realized dataset should be an instance of RealizedObject. If the implementation deviates from the design, then an additional dataset should be indicated as an instance of ExcludedObject. 2. DatasetCharacteristic 3. DataCollectionProcedure 4. DataPreprocesingProcedure 5. IntendedUseCase 6. Limitation 7. Risk 8. Bias 9. Tradeoff 10. IncorrectUseCase 11. IntendedUserGroup 12. HardLaw 13. SoftLaw It is discouraged that the information realization corresponding to the DtasetComponent variable contains the following information elements: 1. SystemAuditMechanism 2. SystemExplanationMechanims 3. SystemHumanOversightMechanism 4. SystemSecurityMechanism 5. SystemIntegrationGuidance 6. SystemDeploymentGuidance 7. SystemDocumentationGuidance 8. SystemIntegrationGuidance 9. SystemUsageGuidance 10. Model 11. ModelCharacteristic 12. Algorithm
A specific type of :Guidance which records a specific piece of information about the guidance to how the :AI_System should be documented.
This concept is imported from ML-Schema to describe the characteristics of the dataset. In RAInS, this includes how many instances make up the dataset, what do the instances represent, what are the types of instances, whether the instances have associated labels or targets, whether the relationships between instances are explicit, as well as the errors, noises, redundancies in the dataset and whether is there any information missing from instances. When used to descriped a Training Dataset, Validation Dataset, or Evaluation Dataset produced by splitting the Dataset Component, it is advisable to describe how the dataset was split, including the ratios of train-validate-test and the class representation percentage in each.
The decision threshold the metric that produced evaluation result was computed on. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A type of :UseCase detailing a situation in which any component of :AI_System or the system as whole should not be used.
A specific type of design specification which represents the description of the purpose(s) for which an AI system is designed and used for. It is advisable that the information realization corresponding to SystemPurposeSpecification contains the following information elements 1. IntendedUserGroup 2. IntendedUseCase 3. IncorrectUseCase
A sao:InformationElement which records a specific piece of information detailing tradeoffs to mitigate possible undesired or dangerous outcomes caused by any aspects of :AI_Sytem.
A relation that links a result to some evaluation measure for which the result was produced.
A sao:InformationElement which records a specific piece of information detailing instructions that should be followed during the deployment or operation stage of the :AI_System lifecycle.
A specific type of a planned human decision representing confirmation of compliance of some sao:AccountableResult (e.g., a model specification) with its compliance requirements.
Hyperparameter is a prior parameter of an implementation, i.e., a parameter which is set before its execution (e.g. C, the complexity parameter, in weka.SMO).
A sao:InformationElement which records a specific piece of information detailing possible undesired or dangerous outcomes caused by any aspects of :AI_Sytem. If a mitigation strategy is used to abate the risk, then these can be indicated using the property :hasMitigationStrategy. It is advisable that information about the risk before and after mitigation is included.
A specific type of planned sao:AccountableResult which represents a high level reference to a specification (e.g. a specification report) that is expected to be produced during the design stage in order to inform the future AI system lifecycle stages (e.g. implementation).
ModelEvaluation is a setting of a value of the performance measure specified by the evaluation specification. It connects a measure specification with its value.
A sao:InformationElement which records information detailing a specific quality of :RealizableObject.
A planned step describing a certification granting decision to be taken by a human when incorporating accountability in AI Systems, e.g. grant an ISO certificate.
A value indicating whether the :RealizableObject is a third party resource.
A sao:InformnationElement which records information describing a certificate (e.g. an ISO certificate) that is granted during the implementation (or deployment or use and maintenance) life cycle stages.
A sao:InformationElement which records information detailing a piece of data, software or hardware that should have been realized but was not. For example, assume the dataset design specification specifies that dataset foorbar is to be realised (to be used to train the model), but the implementation team realises dataset xyzzy instead. In this case, the dataset foobar should be indicated by the implementation team as an Excluded Object - whereas the dataset xyzzy should be created as an instance of :RealizedObject.
A type of :RealizableObjectCharacteristic detailing how :AI_System produces human readable clarifications and justifications of its inner processes and outputs produced.
A sao:InformationElement (which is also an osw:Software and a prov:SoftwareAgent) which records information detailing a software that was used to collect data that formed the dataset which was used to train the ML Model
A type of :RealizableObjectCharacteristic detailing how :AI_System may be safeguarded against adversarial attacks or potential breaches of security.
A specific type of sao:InformationElement which records a specific piece of information detailing how the data is collected. This includes the information about mechanism used for collecting data (e.g. hardware sensors, software program, or manually), the timeframe over which the data was collected, any consent reporting, etc. There is a separate concept that represents the data collection software see rainsDataCollectionSoftware.
A sao:InformationElement (which is also an osw:Software and a prov:SoftwareAgent) which records information detailing a software that was used to preprocess the dataset using to train the ML Model
A specific type of planned ep-plan:Constraint that describes some constraint which cannot be evaluated automatically against the accountability trace. This can be, for example, because the constraint is too high level or difficult to express as a rule. Such constraint would be typically expected to be evaluated manually by a human.
A specific type of a Dataset Component which represents a subset of the whole collected and pre-processed datasets after they were merged then split to be fed into the ML Model. This specific type represents the data used to validate the ML Model. The dataset is typically used repeatedly during the hyperparameter tuning process. The other subsets are used to train and evaluate the model.
A specific type of design specification which represents the specification description of planned requirements, characteristics, performance, etc. related to all the hardware and software 'plumbing' that is not generally considered part of the AI processes, however, may also be source of erroneous behaviour in AI systems. It is advisable that the information realization corresponding to SupportingInfrastructureSpecification contains the following information elements: 1. SupportingInfrastructure 2. SystemAuditMechanism 3. SystemExplanationMechanims 4. SystemHumanOversightMechanism 5. SystemSecurityMechanism 6. Limitation 7. Risk 8. Bias 9. Tradeoff 10. HardLaw 11. SoftLaw
A type of :Risk which records a specific piece of information detailing a potential inclination or prejudice related to some aspects of :AI_Sytem (e.g., a training dataset containing more data about males when both genders will be considered by the ML model) If a mitigation strategy is used to abate the bias, then these can be indicated using the property :hasMitigationStrategy. It is advisable that information about the bias before and after mitigation is included.
A specific type of compliance entity which records a specific piece of information about the procedures, standards, protocols, etc that complying with is desired or encouraged as part of an information realization collection. When used as part of an information realization corresponding to a design specification, it describes the soft laws that are to be complied with. When used as part of an information realization corresponding to an realized component, it describes the soft laws that the component complies with.
A relation that links a realizable object (e.g., model) to the information element describing its characteristic.
A specific type of planned sao:AccountableResult that describes any change from a design specification or guideline. It is expected to be the result of any accountable action that has an input a design specification or a guideline. It is advisable that the information realization corresponding to the Evaluation contains the following information element ChangeJustification.
A value describing the format of an input. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A plan that identifies sao:AccountableAction(s) and sao:AccountableResult(s) relevant to the implementation stage of an AI system and which are required to be documented for accountability purposes. The Accountable Actions that can be done at this stage are: Realize Component, Merge and Split Dataset, Evaluate, Generate Guideline, Grant Certification, and Decide.
Implementation Stage Accountability Plan
A specific type of :Guidance which records a specific piece of information about the guidance to how the :AI_System should be used.
A sao:InformationElement (which is also an osw:Software and a prov:SoftwareAgent) which records information detailing a software that was used to detect bias or check for fairness in both the dataset which was used to train the ML Model as well as the ML model itself
To contact the RAInS team, please send an email to therainsproject [at] gmail [dot] com
-Version 2.0.1: minor edits and additions to the ontology metadata to reflect LOV requirements. -Version 2.0: added classes and relationships to model the Implementation stage of an AI system -Version 1.1: creation, models the Design stage of an AI system
The RAInS Ontology
An ontology to model accountability of AI systems which use machine learning.
A sao:InformationElement which records a specific piece of information detailing some measure used as part of an evaluation process. This is broader than mls:EvaluationMeassure which describes a measure related to a model performance.
A planned step that has some impact on the AI System where the dataset(s) used to produce the ML Model are merged and/or split. Merging takes place when there is more than one dataset in use. Typically, splitting results in three subsets: Training, Validation, and Testing. This step is typically part of the Implementation stage of an AI System's life cycle.
Imported Definition: ModelCharacteristic is a distinguishing quality or property that distinguish one model from another. An example model characteristic may be interpretabilty or a complexity of the model. In RAInS, this includes the type of model, programming method, excluded features, emphasized features, decision thresholds, basis of prioritisation, semantics and similarity functions, editorial criteria, and type of uncertainty. Plus, how those were chosen. There are separate concepts that represent the algorithm and parameters.
A specific type of compliance entity which records a specific piece of information about the laws and regulations that must be, or are, complied with as part of an information realization collection. When used as part of an information realization corresponding to a design specification, it describes the hard laws that must be complied with. When used as part of an information realization corresponding to an realized component, it describes the hard laws that the component complies with.
Model is a generalization of a set of training data able to predict values for unseen instances. It is an output from an execution of a data mining algorithm implementation. Models have a dual nature. They can be treated as data structures and as such represented, stored and manipulated. On the other hand, they act as functions and are executed, taking as input data examples and giving as output the result of applying the function to a data example. Models can also be divided into global or local ones. A global model has global coverage of a data set, i.e., it generalizes the whole data set. A local model, such as a pattern set, is a set of local hypotheses, i.e. each applies to a limited region of the data set.
A specific type of sao:InformationElement which records a specific piece of information detailing how the data is preprocessed. This includes information on what is done to the data (e.g., cleaning, sampling, filtering, feature engineering, etc) and how it is done (e.g., manually, programmatically, or using feature engineering software). There is a separate concept that represents the data preprocessing software.
A sao:InformationElement which records a specific piece of information detailing the result of an evaluation process. This is broader than mls:EvaluationMeassure which describes a result related to a specific model evaluation.
A specific type of :Guidance which records a specific piece of information about the guidance to how the :AI_System should be deployed.
A specific type of design specification which represents the description of a planned evaluation process. It is advisable that the information realization corresponding to EvaluationSpecification contains the following information elements: 1. Either an instance of mls:EvaluationProcedure (if it is the model being evaluated) or EvaluationProcedure (if it is not the ML model being evaluated). 2. Either an instance of mls: EvaluationMeasure (if it is the model being evaluated) or EvaluationMeasure (if it is not the ML model being evaluated). 3. If available: Either an instance of mls:ModelEvaluation (if it is the model being evaluated) or EvaluationResult (if it is not the ML model being evaluated). 4. HardLaw 5. SoftLaw It is discouraged that the information realization corresponding to the Evaluation Specification contains the following information elements: 1. SystemIntegrationGuidance 2. SystemDeploymentGuidance 3. SystemDocumentationGuidance 4. SystemIntegrationGuidance 5. SystemUsageGuidance
A sao:InformationElement which records a specific piece of information detailing the target user audience for the functionality of the overall :AI_System or the components used to develop such system (e.g., a reused public training dataset or a model would be associated with this information).
The lower bound of the confidence interval. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
Imported Definition: The algorithm regardless software implementation.
A value indicating what the version date of a :RealizableObject's is. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
EvaluationProcedure is a technique to evaluate machine learning models. Examples are cross-validation and leave-one-out.
A type of :RealizableObject detailing the hardware and software 'plumbing' that is not generally considered part of the AI processes, however, may also be source of erroneous behaviour in :AI_System.
A sao:InformationElement which records a specific piece of information detailing a situation in which any component of :AI_System or the system as whole could potentially be used.
A specific type of :Guideline which represents the description of guidelines for the Deployment stage of the AI System. It is advisable that the information realization corresponding to the System Deployment Guideline variable contains the following information elements: Guidance. 1. SystemIntegrationGuidance 2. SystemDeploymentGuidance 3. SystemDocumentationGuidance
The AI System for which the accountability plan is specified.
A specific type of design specification which represents the description of a compliance requirement. It is advisable that the information realization corresponding to SystemPurposeSpecification contains the following information elements 1. HardLaw 2. SoftLaw
A specific type of a planned sao:AccountableResult which represents a decision made by a human.
A value indicating what a :RealizableObject's version is. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A specific type of :Evaluation which represents an evaluation performed by a third party independent of the developers of the AI System.
The upper bound of the confidence interval. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A sao:InformationElement which records information detailing a piece of data, software or hardware that was realized - whether or not it followed a design specification or a guideline. For example, assume the dataset design specification specifies that dataset foorbar is to be realised (to be used to train the model), but the implementation team realises dataset xyzzy. In this case, only dataset xyzzy is created as an instance of Realized Object.
A value indicating what an :InformationElement's base 64 image is in String format. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
Inverse of hasRealizableObjectCharacteristic
A plan that identifies sao:AccountableAction(s) and sao:AccountableResult(s) relevant to the design stage of an AI system and which are required to be documented for accountability purposes. The Accountable Actions that can be done at this stage are: Produce Specification and Decide.
A specific type of planned sao:AccountableResult which represents a high level reference to testing at least one component and recording the results of the test(s) (e.g. testing the model with a specified dataset) that is expected to be produced during the implementation stage. It is advisable that the information realization corresponding to the Evaluation variable contains the following information elements: 1. Either an instance of mls:EvaluationProcedure (if it is the model being evaluated) or EvaluationProcedure (if it is not the ML model being evaluated). Either should also be an instance of RealizedObject. If the evaluation deviates from the design, then an additional instance of mls:EvaluationProcedure or EvaluationProcedure should be indicated as an instance of ExcludedObject. 2. Either an instance of mls:EvaluationMeasure (if it is the model being evaluated) or EvaluationMeasure(if it is not the ML model being evaluated). If the evaluation deviates from the design, then an additional instance of mls:EvaluationMeasure or EvaluationMeasure should be indicated as an instance of ExcludedObject. 3. Either an instance of mls:ModelEvaluation (if it is the model being evaluated) or EvaluationResult (if it is not the ML model being evaluated). If the evaluation deviates from the design, then an additional instance of mls:ModelEvaluation or EvaluationResult should be indicated as an instance of ExcludedObject. 4. HardLaw 5. SoftLaw It is discouraged that the information realization corresponding to the Evaluation variable contains the following information elements: 1. SystemIntegrationGuidance 2. SystemDeploymentGuidance 3. SystemDocumentationGuidance 4. SystemIntegrationGuidance 5. SystemUsageGuidance
A value indicating what a :Risk's mitigation strategy is, expressed in String format. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A specific type of a planned human decision representing approvals and sign offs.
A specific type of planned human decision representing confirmation of fitness of some sao:AccountableResult t to its purpose.
A sao:InformationElement which records a specific piece of information detailing procedure by which some aspects of an :AI_Sytem were evaluated. This is broader than mls:EvaluationProcedure which relates to model evaluation.
A planned step describing a decision to be taken by a human when incorporating accountability in AI Systems, e.g. approve the design of an ML Model.
A type of :UseCase detailing a situation in which any component of :AI_System or the system as whole is expected to be used.
A specific type of a Dataset Component which represents a subset of the whole collected and pre-processed datasets after they were merged then split to be fed into the ML Model. This specific type represents the data used to train the ML Model. The other subsets are used to validate and evaluate the model.
A sao:InformationElement which records a specific piece of information detailing a funding source.
EvaluationMeasure is a measure to assess the performance of the model generated by the process that realizes the task. Examples are predictive accuracy or f-measure.
The description of slice that the metric which produced evaluation result was computed on. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A type of :RealizableObjectCharacteristic detailing how :AI_System integrates human operators in its decision processes to offer them the opportunity to stop, review, or alter the outputs produced by the system.
An agent (typically a human) that can be held accountable for his/her actions.
A value describing a note about the version date of a :RealizableObject. Inspired by the Model Card Toolkit schema https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/schema/v0.0.1/model_card.schema.json
A Software is any piece of code that is described in Ontosoft. This could some visualisation software, software package, library, componentised scripts, etc. This can be subclassed if software categories with extra properties are required, otherwise one should use SoftwareTag to categorise software.
A specific type of sao:InformationElement which records a specific piece of information about a compliance requirement that is to be complied with as part of an information realization collection.
A specific type of a Dataset Component which represents a subset of the whole collected and pre-processed datasets after they were merged then split to be fed into the ML Model. This specific type represents the data used to evaluate the ML Model. The other subsets are used to train and evaluate the model. This type can also represent a standalone dataset used to evaluate the model independent of the datasets used to train and validate it.
A sao:InformationElement which records a specific piece of information detailing the limitations of any aspect related to an :AI_System including the system as whole, its components (e.g., ML model), or associated processes (e.g., evaluation).