foundry_dev_tools.utils package

Utils for Foundry DevTools.

Subpackages

• foundry_dev_tools.utils.caches package
  • Submodules
    • foundry_dev_tools.utils.caches.metadata_store module
      • DatasetMetadataStore
    • foundry_dev_tools.utils.caches.spark_caches module
      • DiskPersistenceBackedSparkCache
      • get_dataset_path()
• foundry_dev_tools.utils.converter package
  • Submodules
    • foundry_dev_tools.utils.converter.foundry_spark module
      • foundry_schema_to_spark_schema()
      • spark_schema_to_foundry_schema()
      • infer_dataset_format_from_foundry_schema()
      • foundry_sql_data_to_spark_dataframe()
      • foundry_schema_to_read_options()
      • foundry_schema_to_dataset_format()
      • arrow_stream_to_spark_dataframe()

Submodules

• foundry_dev_tools.utils.api_types module
  • assert_in_literal()
  • Rid
  • TransactionRid
  • DatasetRid
  • RepositoryRid
  • FoundryPath
  • PathInDataset
  • PathInRepository
  • Ref
  • DatasetBranch
  • View
  • RoleId
  • FolderRid
  • JobSpecRid
  • OrganizationRid
  • ProjectRid
  • SourceRid
  • ExtractRid
  • TableRid
  • NetworkEgressPolicyRid
  • FoundryTransaction
  • GroupId
  • SqlDialect
  • SQLReturnType
  • MultipassClientType
  • MultipassGrantType
  • TokenId
  • TokenType
  • UserId
  • CodeResourceType
  • PrincipalTypes
  • Attribution
    • Attribution.time
    • Attribution.user_id
  • CompassBranch
    • CompassBranch.name
    • CompassBranch.rid
    • CompassBranch.urlVariables
    • CompassBranch.classificationRids
  • Branch
    • Branch.id
    • Branch.rid
    • Branch.ancestorBranchIds
    • Branch.creationTime
    • Branch.transactionRid
  • ClassificationBanner
    • ClassificationBanner.classificationString
  • MarkingInfo
    • MarkingInfo.markingId
    • MarkingInfo.markingName
    • MarkingInfo.categoryId
    • MarkingInfo.categoryname
    • MarkingInfo.categoryType
    • MarkingInfo.organizationRid
    • MarkingInfo.isOrganization
    • MarkingInfo.isDirectlyApplied
    • MarkingInfo.disjunctiveInheritedConjunctively
    • MarkingInfo.isCbac
  • BranchWithMarkings
    • BranchWithMarkings.branch
    • BranchWithMarkings.markings
    • BranchWithMarkings.satisfiesConstraints
    • BranchWithMarkings.classificationBanner
  • BackedObjectTypeInfo
    • BackedObjectTypeInfo.id
  • Deprecation
    • Deprecation.message
    • Deprecation.alternativeResourceRid
  • NamedResourceIdentifier
    • NamedResourceIdentifier.rid
    • NamedResourceIdentifier.name
  • LinkedItem
    • LinkedItem.type
    • LinkedItem.rid
    • LinkedItem.attribution
  • Contact
    • Contact.groupId
    • Contact.principalId
  • ContactInformation
    • ContactInformation.primaryContact
  • Classification
    • Classification.userConstraintClassificationBanner
    • Classification.resourceClassificationBanner
  • TransactionMetadata
    • TransactionMetadata.fileCount
    • TransactionMetadata.totalFileSize
    • TransactionMetadata.hiddenFileCount
    • TransactionMetadata.totalHiddenFileSize
  • TransactionRange
    • TransactionRange.startTransactionRid
    • TransactionRange.endTransactionRid
  • ProvenanceRecord
    • ProvenanceRecord.datasetRid
    • ProvenanceRecord.transactionRange
    • ProvenanceRecord.schemaBranchId
    • ProvenanceRecord.schemaVersionId
    • ProvenanceRecord.nonCatalogResources
    • ProvenanceRecord.assumedMarkings
  • NonCatalogProvenanceRecord
    • NonCatalogProvenanceRecord.resources
    • NonCatalogProvenanceRecord.assumedMarkings
    • NonCatalogProvenanceRecord.dependencyType
  • TransactionProvenance
    • TransactionProvenance.provenanceRecords
    • TransactionProvenance.nonCatalogProvenanceRecords
  • Transaction
    • Transaction.type
    • Transaction.status
    • Transaction.filePathType
    • Transaction.startTime
    • Transaction.closeTime
    • Transaction.permissionPath
    • Transaction.record
    • Transaction.attribution
    • Transaction.metadata
    • Transaction.isDataDeleted
    • Transaction.isDeletionComplete
    • Transaction.rid
    • Transaction.provenance
    • Transaction.datasetRid
  • SecuredTransaction
    • SecuredTransaction.transaction
    • SecuredTransaction.rid
  • DatasetIdentity
    • DatasetIdentity.dataset_path
    • DatasetIdentity.dataset_rid
    • DatasetIdentity.last_transaction_rid
    • DatasetIdentity.last_transaction
  • PatchOperation
  • FieldType
  • SortDirection
  • SortSpec
    • SortSpec.field
    • SortSpec.direction
  • PrincipalType
  • Principal
    • Principal.id
    • Principal.type
  • RoleGrant
    • RoleGrant.role
    • RoleGrant.principal
  • RoleGrantPatch
    • RoleGrantPatch.roleGrant
    • RoleGrantPatch.patchOperation
  • UserGroupPrincipalType
  • UserGroupPrincipal
    • UserGroupPrincipal.id
    • UserGroupPrincipal.type
  • UserGroupPrincipalPatch
    • UserGroupPrincipalPatch.principal
    • UserGroupPrincipalPatch.patchOperation
  • ResourceGrantsResult
    • ResourceGrantsResult.grants
    • ResourceGrantsResult.disableInheritedPermissionsForPrincipals
    • ResourceGrantsResult.disableInheritedPermissions
    • ResourceGrantsResult.disableInheritedPermissionsType
  • RoleContext
  • RoleSetId
  • RoleSetUpdate
    • RoleSetUpdate.currentRoleSet
    • RoleSetUpdate.targetRoleSet
    • RoleSetUpdate.rolesMap
  • MarkingPatch
    • MarkingPatch.markingId
    • MarkingPatch.patchOperation
  • CbacMarkingConstraint
    • CbacMarkingConstraint.markingIds
  • MandatoryMarkingConstraintPatchType
  • MandatoryMarkingConstraintPatches
    • MandatoryMarkingConstraintPatches.markingPatches
    • MandatoryMarkingConstraintPatches.type
  • ProjectFolderDisplaySettingsUpdate
    • ProjectFolderDisplaySettingsUpdate.showLinkedOntologyEntitiesTab
    • ProjectFolderDisplaySettingsUpdate.showReleaseAndPublishTab
  • MavenProductId
  • MoveResourcesOption
  • ImportType
  • User
    • User.id
    • User.username
    • User.attributes
  • Group
    • Group.id
    • Group.name
    • Group.attributes
  • GroupMemberExpirationSettings
    • GroupMemberExpirationSettings.maxExpiration
    • GroupMemberExpirationSettings.maxDurationInSeconds
  • DatasetsJobSpecSelection
    • DatasetsJobSpecSelection.datasetRids
    • DatasetsJobSpecSelection.isRequired
  • JobSpecJobSpecsSelection
    • JobSpecJobSpecsSelection.jobSpecRids
    • JobSpecJobSpecsSelection.isRequired
  • UpstreamJobSpecSelection
    • UpstreamJobSpecSelection.datasetRids
    • UpstreamJobSpecSelection.datasetRidsToIgnore
  • ConnectingJobSpecSelection
    • ConnectingJobSpecSelection.upstreamDatasetRids
    • ConnectingJobSpecSelection.downstreamDatasetRids
    • ConnectingJobSpecSelection.datasetRidsToIgnore
  • ExceededDurationMode
  • InputFailureStrategy
  • InputStrategy
    • InputStrategy.inputDatasetRid
    • InputStrategy.failureStrategy
  • OutputQueueStrategy
• foundry_dev_tools.utils.async_s3 module
  • CustomAsyncFoundryCredentialProvider
    • CustomAsyncFoundryCredentialProvider.METHOD
    • CustomAsyncFoundryCredentialProvider.CANONICAL_NAME
    • CustomAsyncFoundryCredentialProvider.__init__()
    • CustomAsyncFoundryCredentialProvider.load()
• foundry_dev_tools.utils.cli module
• foundry_dev_tools.utils.clients module
  • build_api_url()
  • build_public_api_url()
• foundry_dev_tools.utils.compat module
  • v1_to_v2_config_dict()
  • v1_to_v2_config()
  • get_v1_environment_variables()
• foundry_dev_tools.utils.config module
  • cfg_files()
  • user_cfg_files()
  • site_cfg_file()
  • user_cache()
  • merge_dicts()
  • path_from_path_or_str()
  • find_project_config_file()
  • get_environment_variable_config()
  • entry_point_fdt_token_provider()
  • entry_point_fdt_api_client()
  • check_init()
• foundry_dev_tools.utils.misc module
  • decamelize()
  • TailHelper
    • TailHelper.print_handler
    • TailHelper.start_message
    • TailHelper.previous_line_number
    • TailHelper.tail()
    • TailHelper.__init__()
  • print_horizontal_line()
  • is_dataset_a_view()
  • parse_iso()
• foundry_dev_tools.utils.public_api_types module
  • Ontology
    • Ontology.apiname
    • Ontology.displayName
    • Ontology.description
    • Ontology.rid
• foundry_dev_tools.utils.repo module
  • get_repo()
  • get_git_ref()
  • get_git_revision_hash()
  • git_toplevel_dir()
  • get_branch()
• foundry_dev_tools.utils.s3 module
  • CustomFoundryCredentialProvider
    • CustomFoundryCredentialProvider.METHOD
    • CustomFoundryCredentialProvider.CANONICAL_NAME
    • CustomFoundryCredentialProvider.__init__()
    • CustomFoundryCredentialProvider.load()
  • parse_s3_credentials_response()
• foundry_dev_tools.utils.spark module
  • get_spark_session()