Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
BayBE documentation
Light Logo Dark Logo
  • Version: stable

Documentation

  • Readme
  • User Guide
    • Getting Recommendations
    • Campaigns
    • Active Learning
    • Asynchronous Workflows
    • Constraints
    • Environment Vars
    • Insights
    • Objectives
    • Parameters
    • Recommenders
    • Search Spaces
    • Serialization
    • Simulation
    • Surrogates
    • Targets
    • Transfer Learning
    • Utilities
  • Examples
    • Basics
      • Recommenders
      • Campaign
    • Searchspaces
      • Hybrid Space
      • Discrete Space
      • Continuous Space
    • Constraints Discrete
      • Product/Sum
      • Exclusion
      • Custom
      • Dependency
    • Constraints Continuous
      • Hybrid Space
      • Linear
    • Multi Target
      • Desirability
      • Pareto
    • Serialization
      • Basic Serialization
      • Create From Config
      • Validate Config
    • Custom Surrogates
      • Surrogate Params
      • Custom Pretrained
    • Backtesting
      • Custom Blackbox
      • Full Lookup
      • Impute Mode
      • Full Initial Data
    • Custom Hooks
      • Probability Of Improvement
      • Basics
      • Campaign Stopping
    • Transfer Learning
      • Basic Transfer Learning
      • Backtesting
    • Multi Armed Bandit
      • Bernoulli Multi Armed Bandit
    • Mixtures
      • Slot Based
      • Traditional
  • FAQ
  • baybe
    • baybe.acquisition
      • baybe.acquisition.acqfs
        • ExpectedImprovement
        • LogExpectedImprovement
        • PosteriorMean
        • PosteriorStandardDeviation
        • ProbabilityOfImprovement
        • UpperConfidenceBound
        • qExpectedImprovement
        • qKnowledgeGradient
        • qLogExpectedImprovement
        • qLogNParEGO
        • qLogNoisyExpectedHypervolumeImprovement
        • qLogNoisyExpectedImprovement
        • qNegIntegratedPosteriorVariance
        • qNoisyExpectedHypervolumeImprovement
        • qNoisyExpectedImprovement
        • qPosteriorStandardDeviation
        • qProbabilityOfImprovement
        • qSimpleRegret
        • qThompsonSampling
        • qUpperConfidenceBound
      • baybe.acquisition.base
        • AcquisitionFunction
      • baybe.acquisition.partial
        • PartialAcquisitionFunction
      • baybe.acquisition.utils
        • baybe.acquisition.utils.convert_acqf
        • baybe.acquisition.utils.str_to_acqf
    • baybe.campaign
      • Campaign
    • baybe.constraints
      • baybe.constraints.base
        • CardinalityConstraint
        • Constraint
        • ContinuousConstraint
        • ContinuousNonlinearConstraint
        • DiscreteConstraint
      • baybe.constraints.conditions
        • Condition
        • SubSelectionCondition
        • ThresholdCondition
      • baybe.constraints.continuous
        • ContinuousCardinalityConstraint
        • ContinuousLinearConstraint
      • baybe.constraints.deprecation
        • baybe.constraints.deprecation.ContinuousLinearEqualityConstraint
        • baybe.constraints.deprecation.ContinuousLinearInequalityConstraint
        • baybe.constraints.deprecation.structure_constraints
      • baybe.constraints.discrete
        • DiscreteCardinalityConstraint
        • DiscreteCustomConstraint
        • DiscreteDependenciesConstraint
        • DiscreteExcludeConstraint
        • DiscreteLinkedParametersConstraint
        • DiscreteNoLabelDuplicatesConstraint
        • DiscretePermutationInvarianceConstraint
        • DiscreteProductConstraint
        • DiscreteSumConstraint
      • baybe.constraints.utils
        • baybe.constraints.utils.is_cardinality_fulfilled
      • baybe.constraints.validation
        • baybe.constraints.validation.validate_cardinality_constraint_parameter_bounds
        • baybe.constraints.validation.validate_cardinality_constraints_are_nonoverlapping
        • baybe.constraints.validation.validate_constraints
    • baybe.exceptions
      • baybe.exceptions.DeprecationError
      • baybe.exceptions.EmptySearchSpaceError
      • baybe.exceptions.IncompatibilityError
      • baybe.exceptions.IncompatibleAcquisitionFunctionError
      • baybe.exceptions.IncompatibleArgumentError
      • baybe.exceptions.IncompatibleExplainerError
      • baybe.exceptions.IncompatibleSearchSpaceError
      • baybe.exceptions.IncompatibleSurrogateError
      • baybe.exceptions.IncompleteMeasurementsError
      • baybe.exceptions.InfeasibilityError
      • baybe.exceptions.InputDataTypeWarning
      • baybe.exceptions.InvalidSurrogateModelError
      • baybe.exceptions.InvalidTargetValueError
      • baybe.exceptions.MinimumCardinalityViolatedWarning
      • baybe.exceptions.ModelNotTrainedError
      • baybe.exceptions.NoMCAcquisitionFunctionError
      • baybe.exceptions.NoMeasurementsError
      • baybe.exceptions.NoRecommendersLeftError
      • baybe.exceptions.NotEnoughPointsLeftError
      • baybe.exceptions.NothingToSimulateError
      • baybe.exceptions.NumericalUnderflowError
      • baybe.exceptions.OptionalImportError
      • baybe.exceptions.SearchSpaceMatchWarning
      • baybe.exceptions.UnidentifiedSubclassError
      • baybe.exceptions.UnmatchedAttributeError
      • baybe.exceptions.UnusedObjectWarning
    • baybe.insights
      • baybe.insights.shap
        • baybe.insights.shap.SHAP_EXPLAINERS
        • baybe.insights.shap.NON_SHAP_EXPLAINERS
        • baybe.insights.shap.EXPLAINERS
        • baybe.insights.shap.SHAP_PLOTS
        • baybe.insights.shap.is_shap_explainer
        • baybe.insights.shap.make_explainer_for_surrogate
        • SHAPInsight
    • baybe.kernels
      • baybe.kernels.base
        • BasicKernel
        • CompositeKernel
        • Kernel
      • baybe.kernels.basic
        • LinearKernel
        • MaternKernel
        • PeriodicKernel
        • PiecewisePolynomialKernel
        • PolynomialKernel
        • RBFKernel
        • RFFKernel
        • RQKernel
      • baybe.kernels.composite
        • AdditiveKernel
        • ProductKernel
        • ScaleKernel
    • baybe.objectives
      • baybe.objectives.base
        • baybe.objectives.base.to_objective
        • Objective
      • baybe.objectives.desirability
        • baybe.objectives.desirability.scalarize
        • DesirabilityObjective
      • baybe.objectives.enum
        • Scalarizer
      • baybe.objectives.pareto
        • ParetoObjective
      • baybe.objectives.single
        • SingleTargetObjective
      • baybe.objectives.validation
        • baybe.objectives.validation.validate_target_names
    • baybe.parameters
      • baybe.parameters.base
        • ContinuousParameter
        • DiscreteParameter
        • Parameter
      • baybe.parameters.categorical
        • CategoricalParameter
        • TaskParameter
      • baybe.parameters.custom
        • CustomDiscreteParameter
      • baybe.parameters.enum
        • CategoricalEncoding
        • CustomEncoding
        • ParameterEncoding
        • SubstanceEncoding
      • baybe.parameters.numerical
        • NumericalContinuousParameter
        • NumericalDiscreteParameter
      • baybe.parameters.substance
        • baybe.parameters.substance.Smiles
        • SubstanceParameter
      • baybe.parameters.utils
        • baybe.parameters.utils.activate_parameter
        • baybe.parameters.utils.get_parameters_from_dataframe
        • baybe.parameters.utils.is_inactive
        • baybe.parameters.utils.sort_parameters
      • baybe.parameters.validation
        • baybe.parameters.validation.validate_decorrelation
        • baybe.parameters.validation.validate_is_finite
        • baybe.parameters.validation.validate_unique_values
    • baybe.priors
      • baybe.priors.base
        • Prior
      • baybe.priors.basic
        • BetaPrior
        • GammaPrior
        • HalfCauchyPrior
        • HalfNormalPrior
        • LogNormalPrior
        • NormalPrior
        • SmoothedBoxPrior
    • baybe.recommenders
      • baybe.recommenders.base
        • RecommenderProtocol
      • baybe.recommenders.meta
        • baybe.recommenders.meta.base
          • MetaRecommender
        • baybe.recommenders.meta.sequential
          • BaseSequentialMetaRecommender
          • SequentialMetaRecommender
          • StreamingSequentialMetaRecommender
          • TwoPhaseMetaRecommender
      • baybe.recommenders.naive
        • NaiveHybridSpaceRecommender
      • baybe.recommenders.pure
        • baybe.recommenders.pure.base
          • PureRecommender
        • baybe.recommenders.pure.bayesian
          • baybe.recommenders.pure.bayesian.base
            • BayesianRecommender
          • baybe.recommenders.pure.bayesian.botorch
            • BotorchRecommender
          • baybe.recommenders.pure.bayesian.sequential_greedy
            • baybe.recommenders.pure.bayesian.sequential_greedy.SequentialGreedyRecommender
        • baybe.recommenders.pure.nonpredictive
          • baybe.recommenders.pure.nonpredictive.base
            • NonPredictiveRecommender
          • baybe.recommenders.pure.nonpredictive.clustering
            • GaussianMixtureClusteringRecommender
            • KMeansClusteringRecommender
            • PAMClusteringRecommender
            • SKLearnClusteringRecommender
          • baybe.recommenders.pure.nonpredictive.sampling
            • FPSInitialization
            • FPSRecommender
            • RandomRecommender
    • baybe.searchspace
      • baybe.searchspace.continuous
        • SubspaceContinuous
      • baybe.searchspace.core
        • baybe.searchspace.core.to_searchspace
        • baybe.searchspace.core.validate_searchspace_from_config
        • SearchSpace
        • SearchSpaceType
      • baybe.searchspace.discrete
        • baybe.searchspace.discrete.parameter_cartesian_prod_pandas
        • baybe.searchspace.discrete.parameter_cartesian_prod_polars
        • baybe.searchspace.discrete.validate_simplex_subspace_from_config
        • MemorySize
        • SubspaceDiscrete
      • baybe.searchspace.validation
        • baybe.searchspace.validation.get_transform_parameters
        • baybe.searchspace.validation.validate_parameter_names
        • baybe.searchspace.validation.validate_parameters
    • baybe.serialization
      • baybe.serialization.core
        • baybe.serialization.core.converter
        • baybe.serialization.core.block_deserialization_hook
        • baybe.serialization.core.block_serialization_hook
        • baybe.serialization.core.get_base_structure_hook
        • baybe.serialization.core.select_constructor_hook
        • baybe.serialization.core.unstructure_base
      • baybe.serialization.mixin
        • SerialMixin
      • baybe.serialization.utils
        • baybe.serialization.utils.deserialize_dataframe
        • baybe.serialization.utils.serialize_dataframe
    • baybe.simulation
      • baybe.simulation.core
        • baybe.simulation.core.simulate_experiment
      • baybe.simulation.lookup
        • baybe.simulation.lookup.look_up_targets
      • baybe.simulation.scenarios
        • baybe.simulation.scenarios.simulate_scenarios
      • baybe.simulation.transfer_learning
        • baybe.simulation.transfer_learning.simulate_transfer_learning
    • baybe.surrogates
      • baybe.surrogates.bandit
        • BetaBernoulliMultiArmedBanditSurrogate
      • baybe.surrogates.base
        • baybe.surrogates.base.PosteriorStatistic
        • IndependentGaussianSurrogate
        • Surrogate
        • SurrogateProtocol
      • baybe.surrogates.composite
        • CompositeSurrogate
      • baybe.surrogates.custom
        • baybe.surrogates.custom.register_custom_architecture
        • CustomONNXSurrogate
      • baybe.surrogates.gaussian_process
        • baybe.surrogates.gaussian_process.core
          • GaussianProcessSurrogate
        • baybe.surrogates.gaussian_process.kernel_factory
          • baybe.surrogates.gaussian_process.kernel_factory.to_kernel_factory
          • KernelFactory
          • PlainKernelFactory
        • baybe.surrogates.gaussian_process.presets
          • baybe.surrogates.gaussian_process.presets.core
            • baybe.surrogates.gaussian_process.presets.core.make_gp_from_preset
            • GaussianProcessPreset
          • baybe.surrogates.gaussian_process.presets.default
            • DefaultKernelFactory
          • baybe.surrogates.gaussian_process.presets.edbo
            • EDBOKernelFactory
      • baybe.surrogates.linear
        • BayesianLinearSurrogate
      • baybe.surrogates.naive
        • MeanPredictionSurrogate
      • baybe.surrogates.ngboost
        • NGBoostSurrogate
      • baybe.surrogates.random_forest
        • RandomForestSurrogate
      • baybe.surrogates.utils
        • baybe.surrogates.utils.batchify_ensemble_predictor
        • baybe.surrogates.utils.batchify_mean_var_prediction
        • baybe.surrogates.utils.catch_constant_targets
      • baybe.surrogates.validation
        • baybe.surrogates.validation.type_validation_converter
        • baybe.surrogates.validation.make_dict_validator
        • baybe.surrogates.validation.validate_custom_architecture_cls
    • baybe.targets
      • baybe.targets.base
        • Target
      • baybe.targets.binary
        • baybe.targets.binary.ChoiceValue
        • BinaryTarget
      • baybe.targets.enum
        • TargetMode
        • TargetTransformation
      • baybe.targets.numerical
        • NumericalTarget
      • baybe.targets.transforms
        • baybe.targets.transforms.bell_transform
        • baybe.targets.transforms.linear_transform
        • baybe.targets.transforms.triangular_transform
    • baybe.telemetry
      • baybe.telemetry.get_user_details
      • baybe.telemetry.is_enabled
      • baybe.telemetry.telemetry_record_recommended_measurement_percentage
      • baybe.telemetry.telemetry_record_value
    • baybe.utils
      • baybe.utils.augmentation
        • baybe.utils.augmentation.df_apply_dependency_augmentation
        • baybe.utils.augmentation.df_apply_permutation_augmentation
      • baybe.utils.basic
        • baybe.utils.basic.UNSPECIFIED
        • baybe.utils.basic.convert_to_float
        • baybe.utils.basic.find_subclass
        • baybe.utils.basic.get_baseclasses
        • baybe.utils.basic.get_subclasses
        • baybe.utils.basic.group_duplicate_values
        • baybe.utils.basic.hilberts_factory
        • baybe.utils.basic.is_all_instance
        • baybe.utils.basic.match_attributes
        • baybe.utils.basic.refers_to
        • baybe.utils.basic.register_hooks
        • baybe.utils.basic.to_tuple
        • Dummy
        • UnspecifiedType
        • classproperty
      • baybe.utils.boolean
        • baybe.utils.boolean.check_if_in
        • baybe.utils.boolean.is_abstract
        • baybe.utils.boolean.strtobool
      • baybe.utils.chemistry
        • baybe.utils.chemistry.get_canonical_smiles
        • baybe.utils.chemistry.get_fingerprint_class
        • baybe.utils.chemistry.name_to_smiles
        • baybe.utils.chemistry.smiles_to_fingerprint_features
      • baybe.utils.clustering_algorithms
        • baybe.utils.clustering_algorithms.third_party
          • baybe.utils.clustering_algorithms.third_party.kmedoids
            • KMedoids
      • baybe.utils.conversion
        • baybe.utils.conversion.fraction_to_float
        • baybe.utils.conversion.nonstring_to_tuple
        • baybe.utils.conversion.to_string
      • baybe.utils.dataframe
        • baybe.utils.dataframe.add_fake_measurements
        • baybe.utils.dataframe.add_noise_to_perturb_degenerate_rows
        • baybe.utils.dataframe.add_parameter_noise
        • baybe.utils.dataframe.arrays_to_dataframes
        • baybe.utils.dataframe.create_fake_input
        • baybe.utils.dataframe.df_drop_single_value_columns
        • baybe.utils.dataframe.df_drop_string_columns
        • baybe.utils.dataframe.df_uncorrelated_features
        • baybe.utils.dataframe.filter_df
        • baybe.utils.dataframe.fuzzy_row_match
        • baybe.utils.dataframe.get_transform_objects
        • baybe.utils.dataframe.handle_missing_values
        • baybe.utils.dataframe.normalize_input_dtypes
        • baybe.utils.dataframe.pretty_print_df
        • baybe.utils.dataframe.to_tensor
        • baybe.utils.dataframe.transform_target_columns
      • baybe.utils.interval
        • baybe.utils.interval.convert_bounds
        • baybe.utils.interval.use_fallback_constructor_hook
        • Interval
        • baybe.utils.interval.InfiniteIntervalError
      • baybe.utils.memory
        • baybe.utils.memory.bytes_to_human_readable
      • baybe.utils.numerical
        • baybe.utils.numerical.VARNAME_NUMPY_USE_SINGLE_PRECISION
        • baybe.utils.numerical.DTypeFloatNumpy
        • baybe.utils.numerical.DTypeFloatONNX
        • baybe.utils.numerical.closer_element
        • baybe.utils.numerical.closest_element
        • baybe.utils.numerical.geom_mean
      • baybe.utils.random
        • baybe.utils.random.set_random_seed
        • baybe.utils.random.temporary_seed
      • baybe.utils.sampling_algorithms
        • baybe.utils.sampling_algorithms.farthest_point_sampling
        • baybe.utils.sampling_algorithms.sample_numerical_df
        • DiscreteSamplingMethod
      • baybe.utils.scaling
        • ColumnTransformer
      • baybe.utils.torch
        • baybe.utils.torch.VARNAME_TORCH_USE_SINGLE_PRECISION
        • baybe.utils.torch.DTypeFloatTorch
      • baybe.utils.validation
        • baybe.utils.validation.finite_float
        • baybe.utils.validation.non_nan_float
        • baybe.utils.validation.non_inf_float
        • baybe.utils.validation.finite_float
        • baybe.utils.validation.non_inf_float
        • baybe.utils.validation.non_nan_float
        • baybe.utils.validation.validate_not_nan
        • baybe.utils.validation.validate_parameter_input
        • baybe.utils.validation.validate_target_input

Misc

  • Contribute
  • Contributors
  • Known Issues
  • Changelog
  • Github
  • License
Back to top
View this page

Backtesting¶

These examples demonstrate BayBE’s Backtesting capabilities.

  • Custom Blackbox
  • Full Lookup
  • Impute Mode
  • Full Initial Data
Next
Optimizing a Custom Black-Box Function
Previous
Example for surrogate model with a custom pretrained model
Copyright © 2022-2025 Merck KGaA, Darmstadt, Germany and/or its affiliates. All rights reserved.
Made with Sphinx and @pradyunsg's Furo