Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
You are not viewing the documentation of the stable version.
BayBE documentation
Light Logo Dark Logo
  • Version: latest

User Guide

  • Readme
  • Concepts
    • Active Learning
    • Asynchronous Workflows
    • Getting Recommendations
    • Serialization
    • Settings
    • Simulation
    • Transfer Learning
  • Components
    • Campaigns
    • Constraints
    • Insights
    • Objectives
    • Parameters
    • Recommenders
    • Search Spaces
    • Surrogates
    • Targets
    • Transformations
    • Utilities
  • Examples
    • Basics
      • Recommenders
      • Campaign
    • Searchspaces
      • Discrete Space
      • Hybrid Space
      • Continuous Space
    • Constraints Discrete
      • Dependency
      • Custom
      • Product/Sum
      • Exclusion
    • Constraints Continuous
      • Linear
      • Hybrid Space
      • Interpoint
    • Multi Target
      • Pareto
      • Desirability
    • Serialization
      • Validate Config
      • Create From Config
      • Basic Serialization
    • Custom Surrogates
      • Custom Pretrained
      • Surrogate Params
    • Mixtures
      • Slot Based
      • Traditional
    • Custom Hooks
      • Probability Of Improvement
      • Campaign Stopping
      • Basics
    • Multi Armed Bandit
      • Bernoulli Multi Armed Bandit
    • Backtesting
      • Custom Blackbox
      • Full Initial Data
      • Impute Mode
      • Full Lookup
    • Transformations
      • Laser Tuning
    • Transfer Learning
      • Basic Transfer Learning
  • API Reference
    • baybe.acquisition
      • baybe.acquisition.acqfs
        • ExpectedImprovement
        • LogExpectedImprovement
        • PosteriorMean
        • PosteriorStandardDeviation
        • ProbabilityOfImprovement
        • UpperConfidenceBound
        • qExpectedHypervolumeImprovement
        • qExpectedImprovement
        • qKnowledgeGradient
        • qLogExpectedHypervolumeImprovement
        • qLogExpectedImprovement
        • qLogNParEGO
        • qLogNoisyExpectedHypervolumeImprovement
        • qLogNoisyExpectedImprovement
        • qNegIntegratedPosteriorVariance
        • qNoisyExpectedHypervolumeImprovement
        • qNoisyExpectedImprovement
        • qPosteriorStandardDeviation
        • qProbabilityOfImprovement
        • qSimpleRegret
        • qThompsonSampling
        • qUpperConfidenceBound
      • baybe.acquisition.base
        • AcquisitionFunction
      • baybe.acquisition.partial
        • PartialAcquisitionFunction
      • baybe.acquisition.utils
        • baybe.acquisition.utils.convert_acqf
        • baybe.acquisition.utils.make_partitioning
        • baybe.acquisition.utils.str_to_acqf
    • baybe.campaign
      • Campaign
    • baybe.constraints
      • baybe.constraints.base
        • CardinalityConstraint
        • Constraint
        • ContinuousConstraint
        • ContinuousNonlinearConstraint
        • DiscreteConstraint
      • baybe.constraints.conditions
        • Condition
        • SubSelectionCondition
        • ThresholdCondition
      • baybe.constraints.continuous
        • ContinuousCardinalityConstraint
        • ContinuousLinearConstraint
      • baybe.constraints.discrete
        • DiscreteBatchConstraint
        • DiscreteCardinalityConstraint
        • DiscreteCustomConstraint
        • DiscreteDependenciesConstraint
        • DiscreteExcludeConstraint
        • DiscreteLinkedParametersConstraint
        • DiscreteNoLabelDuplicatesConstraint
        • DiscretePermutationInvarianceConstraint
        • DiscreteProductConstraint
        • DiscreteSumConstraint
      • baybe.constraints.utils
        • baybe.constraints.utils.is_cardinality_fulfilled
      • baybe.constraints.validation
        • baybe.constraints.validation.validate_cardinality_constraint_parameter_bounds
        • baybe.constraints.validation.validate_cardinality_constraints_are_nonoverlapping
        • baybe.constraints.validation.validate_constraints
    • baybe.exceptions
      • baybe.exceptions.DeprecationError
      • baybe.exceptions.EmptySearchSpaceError
      • baybe.exceptions.IncompatibilityError
      • baybe.exceptions.IncompatibleAcquisitionFunctionError
      • baybe.exceptions.IncompatibleArgumentError
      • baybe.exceptions.IncompatibleExplainerError
      • baybe.exceptions.IncompatibleSearchSpaceError
      • baybe.exceptions.IncompatibleSurrogateError
      • baybe.exceptions.IncompleteMeasurementsError
      • baybe.exceptions.InfeasibilityError
      • baybe.exceptions.InputDataTypeWarning
      • baybe.exceptions.InvalidTargetValueError
      • baybe.exceptions.MinimumCardinalityViolatedWarning
      • baybe.exceptions.ModelNotTrainedError
      • baybe.exceptions.NoMCAcquisitionFunctionError
      • baybe.exceptions.NoMeasurementsError
      • baybe.exceptions.NoRecommendersLeftError
      • baybe.exceptions.NonGaussianityError
      • baybe.exceptions.NotAllowedError
      • baybe.exceptions.NotEnoughPointsLeftError
      • baybe.exceptions.NothingToComputeError
      • baybe.exceptions.NothingToSimulateError
      • baybe.exceptions.NumericalUnderflowError
      • baybe.exceptions.OptionalImportError
      • baybe.exceptions.SearchSpaceMatchWarning
      • baybe.exceptions.UnidentifiedSubclassError
      • baybe.exceptions.UnmatchedAttributeError
      • baybe.exceptions.UnsupportedEarlyFilteringError
      • baybe.exceptions.UnusedObjectWarning
    • baybe.insights
      • baybe.insights.shap
        • baybe.insights.shap.SHAP_EXPLAINERS
        • baybe.insights.shap.NON_SHAP_EXPLAINERS
        • baybe.insights.shap.EXPLAINERS
        • baybe.insights.shap.SHAP_PLOTS
        • baybe.insights.shap.is_shap_explainer
        • baybe.insights.shap.make_explainer_for_surrogate
        • SHAPInsight
    • baybe.kernels
      • baybe.kernels.base
        • BasicKernel
        • CompositeKernel
        • Kernel
      • baybe.kernels.basic
        • IndexKernel
        • LinearKernel
        • MaternKernel
        • PeriodicKernel
        • PiecewisePolynomialKernel
        • PolynomialKernel
        • PositiveIndexKernel
        • RBFKernel
        • RFFKernel
        • RQKernel
      • baybe.kernels.composite
        • AdditiveKernel
        • ProductKernel
        • ScaleKernel
    • baybe.objectives
      • baybe.objectives.base
        • baybe.objectives.base.to_objective
        • Objective
      • baybe.objectives.botorch
        • ChainedMCObjective
      • baybe.objectives.desirability
        • DesirabilityObjective
      • baybe.objectives.enum
        • Scalarizer
      • baybe.objectives.pareto
        • ParetoObjective
      • baybe.objectives.single
        • SingleTargetObjective
      • baybe.objectives.validation
        • baybe.objectives.validation.validate_target_names
    • baybe.parameters
      • baybe.parameters.base
        • ContinuousParameter
        • DiscreteParameter
        • Parameter
      • baybe.parameters.categorical
        • CategoricalParameter
        • TaskParameter
      • baybe.parameters.custom
        • CustomDiscreteParameter
      • baybe.parameters.enum
        • CategoricalEncoding
        • CustomEncoding
        • ParameterEncoding
        • SubstanceEncoding
      • baybe.parameters.numerical
        • NumericalContinuousParameter
        • NumericalDiscreteParameter
      • baybe.parameters.selectors
        • baybe.parameters.selectors.to_parameter_selector
        • NameSelector
        • ParameterSelector
        • ParameterSelectorProtocol
        • TypeSelector
      • baybe.parameters.substance
        • baybe.parameters.substance.Smiles
        • SubstanceParameter
      • baybe.parameters.utils
        • baybe.parameters.utils.activate_parameter
        • baybe.parameters.utils.get_parameters_from_dataframe
        • baybe.parameters.utils.is_inactive
        • baybe.parameters.utils.sort_parameters
      • baybe.parameters.validation
        • baybe.parameters.validation.validate_decorrelation
        • baybe.parameters.validation.validate_is_finite
        • baybe.parameters.validation.validate_unique_values
    • baybe.priors
      • baybe.priors.base
        • Prior
      • baybe.priors.basic
        • BetaPrior
        • GammaPrior
        • HalfCauchyPrior
        • HalfNormalPrior
        • LogNormalPrior
        • NormalPrior
        • SmoothedBoxPrior
    • baybe.recommenders
      • baybe.recommenders.base
        • RecommenderProtocol
      • baybe.recommenders.meta
        • baybe.recommenders.meta.base
          • MetaRecommender
        • baybe.recommenders.meta.sequential
          • BaseSequentialMetaRecommender
          • SequentialMetaRecommender
          • StreamingSequentialMetaRecommender
          • TwoPhaseMetaRecommender
      • baybe.recommenders.naive
        • NaiveHybridSpaceRecommender
      • baybe.recommenders.pure
        • baybe.recommenders.pure.base
          • PureRecommender
        • baybe.recommenders.pure.bayesian
          • baybe.recommenders.pure.bayesian.base
            • BayesianRecommender
          • baybe.recommenders.pure.bayesian.botorch
            • baybe.recommenders.pure.bayesian.botorch.continuous
              • baybe.recommenders.pure.bayesian.botorch.continuous.recommend_continuous_torch
              • baybe.recommenders.pure.bayesian.botorch.continuous.recommend_continuous_with_cardinality_constraints
              • baybe.recommenders.pure.bayesian.botorch.continuous.recommend_continuous_without_cardinality_constraints
            • baybe.recommenders.pure.bayesian.botorch.core
              • BotorchRecommender
            • baybe.recommenders.pure.bayesian.botorch.discrete
              • baybe.recommenders.pure.bayesian.botorch.discrete.recommend_discrete_with_subsets
              • baybe.recommenders.pure.bayesian.botorch.discrete.recommend_discrete_without_subsets
            • baybe.recommenders.pure.bayesian.botorch.hybrid
              • baybe.recommenders.pure.bayesian.botorch.hybrid.recommend_hybrid_with_subsets
              • baybe.recommenders.pure.bayesian.botorch.hybrid.recommend_hybrid_without_subsets
        • baybe.recommenders.pure.nonpredictive
          • baybe.recommenders.pure.nonpredictive.base
            • NonPredictiveRecommender
          • baybe.recommenders.pure.nonpredictive.clustering
            • GaussianMixtureClusteringRecommender
            • KMeansClusteringRecommender
            • PAMClusteringRecommender
            • SKLearnClusteringRecommender
          • baybe.recommenders.pure.nonpredictive.sampling
            • FPSInitialization
            • FPSRecommender
            • RandomRecommender
    • baybe.searchspace
      • baybe.searchspace.continuous
        • SubspaceContinuous
      • baybe.searchspace.core
        • baybe.searchspace.core.to_searchspace
        • baybe.searchspace.core.validate_searchspace_from_config
        • SearchSpace
        • SearchSpaceType
      • baybe.searchspace.discrete
        • baybe.searchspace.discrete.validate_simplex_subspace_from_config
        • MemorySize
        • SubspaceDiscrete
      • baybe.searchspace.utils
        • baybe.searchspace.utils.build_constrained_product
        • baybe.searchspace.utils.optimize_parameter_order
        • baybe.searchspace.utils.parameter_cartesian_prod_pandas
        • baybe.searchspace.utils.parameter_cartesian_prod_pandas_constrained
        • baybe.searchspace.utils.parameter_cartesian_prod_polars
        • baybe.searchspace.utils.select_via_flat_index
      • baybe.searchspace.validation
        • baybe.searchspace.validation.get_transform_parameters
        • baybe.searchspace.validation.validate_dataframe_active_values
        • baybe.searchspace.validation.validate_parameter_names
        • baybe.searchspace.validation.validate_parameters
    • baybe.serialization
      • baybe.serialization.core
        • baybe.serialization.core.converter
        • baybe.serialization.core.add_type
        • baybe.serialization.core.block_deserialization_hook
        • baybe.serialization.core.block_serialization_hook
        • baybe.serialization.core.make_base_structure_hook
        • baybe.serialization.core.make_base_unstructure_hook
        • baybe.serialization.core.select_constructor_hook
        • baybe.serialization.core.unstructure_with_type
      • baybe.serialization.mixin
        • SerialMixin
        • SupportsRead
        • SupportsWrite
      • baybe.serialization.utils
        • baybe.serialization.utils.deserialize_dataframe
        • baybe.serialization.utils.serialize_dataframe
    • baybe.settings
      • baybe.settings.active_settings
      • baybe.settings.adjust_defaults
      • Settings
    • baybe.simulation
      • baybe.simulation.core
        • baybe.simulation.core.simulate_experiment
      • baybe.simulation.lookup
        • baybe.simulation.lookup.look_up_targets
      • baybe.simulation.scenarios
        • baybe.simulation.scenarios.simulate_scenarios
      • baybe.simulation.transfer_learning
        • baybe.simulation.transfer_learning.simulate_transfer_learning
    • baybe.surrogates
      • baybe.surrogates.bandit
        • BetaBernoulliMultiArmedBanditSurrogate
      • baybe.surrogates.base
        • baybe.surrogates.base.PosteriorStatistic
        • IndependentGaussianSurrogate
        • Surrogate
        • SurrogateProtocol
      • baybe.surrogates.composite
        • CompositeSurrogate
      • baybe.surrogates.custom
        • CustomONNXSurrogate
      • baybe.surrogates.gaussian_process
        • baybe.surrogates.gaussian_process.components
          • baybe.surrogates.gaussian_process.components.fit_criterion
            • baybe.surrogates.gaussian_process.components.fit_criterion.FitCriterionFactoryProtocol
            • baybe.surrogates.gaussian_process.components.fit_criterion.PlainFitCriterionFactory
            • FitCriterion
          • baybe.surrogates.gaussian_process.components.generic
            • baybe.surrogates.gaussian_process.components.generic.to_component_factory
            • GPComponentFactoryProtocol
            • GPComponentType
            • PlainGPComponentFactory
          • baybe.surrogates.gaussian_process.components.kernel
            • baybe.surrogates.gaussian_process.components.kernel.KernelFactoryProtocol
            • baybe.surrogates.gaussian_process.components.kernel.PlainKernelFactory
            • ICMKernelFactory
          • baybe.surrogates.gaussian_process.components.likelihood
            • baybe.surrogates.gaussian_process.components.likelihood.LikelihoodFactoryProtocol
            • baybe.surrogates.gaussian_process.components.likelihood.PlainLikelihoodFactory
            • LazyGaussianLikelihoodFactory
          • baybe.surrogates.gaussian_process.components.mean
            • baybe.surrogates.gaussian_process.components.mean.MeanFactoryProtocol
            • baybe.surrogates.gaussian_process.components.mean.PlainMeanFactory
            • LazyConstantMeanFactory
        • baybe.surrogates.gaussian_process.core
          • GaussianProcessSurrogate
        • baybe.surrogates.gaussian_process.presets
          • baybe.surrogates.gaussian_process.presets.baybe
            • BayBEFitCriterionFactory
            • BayBEKernelFactory
            • BayBELikelihoodFactory
            • BayBEMeanFactory
          • baybe.surrogates.gaussian_process.presets.botorch
          • baybe.surrogates.gaussian_process.presets.chen
            • ChenKernelFactory
          • baybe.surrogates.gaussian_process.presets.core
            • GaussianProcessPreset
          • baybe.surrogates.gaussian_process.presets.edbo
            • EDBOKernelFactory
            • EDBOLikelihoodFactory
            • EDBOMeanFactory
          • baybe.surrogates.gaussian_process.presets.edbo_smoothed
            • SmoothedEDBOKernelFactory
            • SmoothedEDBOKernelFactory
            • SmoothedEDBOLikelihoodFactory
            • SmoothedEDBOMeanFactory
          • baybe.surrogates.gaussian_process.presets.hvarfner
            • HvarfnerKernelFactory
            • HvarfnerLikelihoodFactory
            • HvarfnerMeanFactory
      • baybe.surrogates.linear
        • BayesianLinearSurrogate
      • baybe.surrogates.naive
        • MeanPredictionSurrogate
      • baybe.surrogates.ngboost
        • NGBoostSurrogate
      • baybe.surrogates.random_forest
        • RandomForestSurrogate
      • baybe.surrogates.utils
        • baybe.surrogates.utils.batchify_ensemble_predictor
        • baybe.surrogates.utils.batchify_mean_var_prediction
        • baybe.surrogates.utils.catch_constant_targets
      • baybe.surrogates.validation
        • baybe.surrogates.validation.type_validation_converter
        • baybe.surrogates.validation.make_dict_validator
        • baybe.surrogates.validation.validate_custom_architecture_cls
    • baybe.targets
      • baybe.targets.base
        • Target
      • baybe.targets.binary
        • baybe.targets.binary.ChoiceValue
        • BinaryTarget
      • baybe.targets.enum
        • MatchMode
      • baybe.targets.numerical
        • NumericalTarget
      • baybe.targets.transforms
      • baybe.targets.utils
        • baybe.targets.utils.capture_constructor_info
        • baybe.targets.utils.combine_numerical_targets
    • baybe.transformations
      • baybe.transformations.base
        • MonotonicTransformation
        • Transformation
      • baybe.transformations.basic
        • AbsoluteTransformation
        • AffineTransformation
        • BellTransformation
        • ClampingTransformation
        • CustomTransformation
        • ExponentialTransformation
        • IdentityTransformation
        • LogarithmicTransformation
        • PowerTransformation
        • SigmoidTransformation
        • TriangularTransformation
        • TwoSidedAffineTransformation
      • baybe.transformations.composite
        • AdditiveTransformation
        • ChainedTransformation
        • MultiplicativeTransformation
      • baybe.transformations.utils
        • baybe.transformations.utils.combine_affine_transformations
        • baybe.transformations.utils.compress_transformations
        • baybe.transformations.utils.convert_transformation
        • baybe.transformations.utils.sum_affine_transformations
    • baybe.utils
      • baybe.utils.augmentation
        • baybe.utils.augmentation.df_apply_dependency_augmentation
        • baybe.utils.augmentation.df_apply_permutation_augmentation
      • baybe.utils.basic
        • baybe.utils.basic.UNSPECIFIED
        • baybe.utils.basic.cache_to_disk
        • baybe.utils.basic.compose
        • baybe.utils.basic.compose_two
        • baybe.utils.basic.convert_to_float
        • baybe.utils.basic.find_subclass
        • baybe.utils.basic.flatten
        • baybe.utils.basic.get_baseclasses
        • baybe.utils.basic.get_subclasses
        • baybe.utils.basic.group_duplicate_values
        • baybe.utils.basic.hilberts_factory
        • baybe.utils.basic.is_all_instance
        • baybe.utils.basic.match_attributes
        • baybe.utils.basic.refers_to
        • baybe.utils.basic.register_hooks
        • baybe.utils.basic.to_tuple
        • UnspecifiedType
        • classproperty
      • baybe.utils.boolean
        • baybe.utils.boolean.check_if_in
        • baybe.utils.boolean.is_abstract
        • baybe.utils.boolean.strtobool
        • baybe.utils.boolean.structure_autobool
        • baybe.utils.boolean.to_bool
        • baybe.utils.boolean.unstructure_autobool
        • AutoBool
        • UncertainBool
      • baybe.utils.chemistry
        • baybe.utils.chemistry.get_canonical_smiles
        • baybe.utils.chemistry.get_fingerprint_class
        • baybe.utils.chemistry.name_to_smiles
        • baybe.utils.chemistry.smiles_to_fingerprint_features
      • baybe.utils.clustering_algorithms
        • baybe.utils.clustering_algorithms.third_party
          • baybe.utils.clustering_algorithms.third_party.kmedoids
            • KMedoids
      • baybe.utils.conversion
        • baybe.utils.conversion.fraction_to_float
        • baybe.utils.conversion.nonstring_to_tuple
        • baybe.utils.conversion.to_string
      • baybe.utils.dataframe
        • baybe.utils.dataframe.add_fake_measurements
        • baybe.utils.dataframe.add_noise_to_perturb_degenerate_rows
        • baybe.utils.dataframe.add_parameter_noise
        • baybe.utils.dataframe.arrays_to_dataframes
        • baybe.utils.dataframe.create_fake_input
        • baybe.utils.dataframe.df_drop_single_value_columns
        • baybe.utils.dataframe.df_drop_string_columns
        • baybe.utils.dataframe.df_uncorrelated_features
        • baybe.utils.dataframe.filter_df
        • baybe.utils.dataframe.fuzzy_row_match
        • baybe.utils.dataframe.get_transform_objects
        • baybe.utils.dataframe.handle_missing_values
        • baybe.utils.dataframe.normalize_input_dtypes
        • baybe.utils.dataframe.pretty_print_df
        • baybe.utils.dataframe.to_tensor
        • baybe.utils.dataframe.transform_target_columns
      • baybe.utils.interval
        • baybe.utils.interval.ConvertibleToInterval
        • baybe.utils.interval.use_fallback_constructor_hook
        • Interval
        • baybe.utils.interval.InfiniteIntervalError
      • baybe.utils.memory
        • baybe.utils.memory.bytes_to_human_readable
      • baybe.utils.metadata
        • baybe.utils.metadata.ConvertibleToMeasurableMetadata
        • baybe.utils.metadata.to_metadata
        • MeasurableMetadata
        • Metadata
      • baybe.utils.numerical
        • baybe.utils.numerical.DTypeFloatONNX
        • baybe.utils.numerical.closer_element
        • baybe.utils.numerical.closest_element
      • baybe.utils.random
        • baybe.utils.random.set_random_seed
        • baybe.utils.random.temporary_seed
      • baybe.utils.sampling_algorithms
        • baybe.utils.sampling_algorithms.farthest_point_sampling
        • baybe.utils.sampling_algorithms.sample_numerical_df
        • DiscreteSamplingMethod
      • baybe.utils.scaling
        • ColumnTransformer
      • baybe.utils.torch
        • baybe.utils.torch.torch_to_numpy_dtype_mapping
      • baybe.utils.validation
        • baybe.utils.validation.finite_float
        • baybe.utils.validation.non_nan_float
        • baybe.utils.validation.non_inf_float
        • baybe.utils.validation.finite_float
        • baybe.utils.validation.non_inf_float
        • baybe.utils.validation.non_nan_float
        • baybe.utils.validation.preprocess_dataframe
        • baybe.utils.validation.validate_not_nan
        • baybe.utils.validation.validate_object_names
        • baybe.utils.validation.validate_objective_input
        • baybe.utils.validation.validate_parameter_input
        • baybe.utils.validation.validate_target_input
  • FAQ

Project

  • Contribute
  • Contributors
  • Known Issues
  • Changelog
  • References
  • Github
  • License
Back to top
View this page

Backtesting¶

These examples demonstrate BayBE’s Backtesting capabilities.

  • Custom Blackbox
  • Full Initial Data
  • Impute Mode
  • Full Lookup
Next
Optimizing a Custom Black-Box Function
Previous
Bernoulli Multi Armed Bandit
Copyright © 2022-2025 Merck KGaA, Darmstadt, Germany and/or its affiliates. All rights reserved.
Made with Sphinx and @pradyunsg's Furo