foundry_dev_tools.clients.foundry_sql_server module

foundry_dev_tools.clients.foundry_sql_server module#

Implementation of the foundry-sql-server API.

class foundry_dev_tools.clients.foundry_sql_server.FoundrySqlServerClient[source]#

Bases: APIClient

FoundrySqlServerClient class that implements methods from the ‘foundry-sql-server’ API.

api_name: ClassVar[str] = 'foundry-sql-server'#
query_foundry_sql(query: str, return_type: Literal['pandas'], branch: Ref = 'master', sql_dialect: SqlDialect = 'SPARK', timeout: int = 600) → pd.core.frame.DataFrame[source]#
query_foundry_sql(query: str, return_type: Literal['spark'], branch: Ref = 'master', sql_dialect: SqlDialect = 'SPARK', timeout: int = 600) → pyspark.sql.DataFrame
query_foundry_sql(query: str, return_type: Literal['arrow'], branch: Ref = 'master', sql_dialect: SqlDialect = 'SPARK', timeout: int = 600) → pa.Table
query_foundry_sql(query: str, return_type: Literal['raw'], branch: Ref = 'master', sql_dialect: SqlDialect = 'SPARK', timeout: int = 600) → tuple[dict, list[list]]
query_foundry_sql(query: str, return_type: SQLReturnType = 'pandas', branch: Ref = 'master', sql_dialect: SqlDialect = 'SPARK', timeout: int = 600) → tuple[dict, list[list]] | pd.core.frame.DataFrame | pa.Table | pyspark.sql.DataFrame

Queries the Foundry SQL server with spark SQL dialect.

Uses Arrow IPC to communicate with the Foundry SQL Server Endpoint.

Falls back to query_foundry_sql_legacy in case pyarrow is not installed or the query does not return Arrow Format.

Example

df1 = client.query_foundry_sql(“SELECT * FROM /Global/Foundry Operations/Foundry Support/iris”) query = (“SELECT col1 FROM {start_transaction_rid}:{end_transaction_rid}@{branch}.`{dataset_path_or_rid}` WHERE filterColumns = ‘value1’ LIMIT 1”) df2 = client.query_foundry_sql(query)

Parameters:

  • query – The SQL Query in Foundry Spark Dialect (use backticks instead of quotes)

  • branch – the dataset branch

  • sql_dialect – the sql dialect

  • return_type – See :py:class:foundry_dev_tools.foundry_api_client.SQLReturnType

  • timeout – Query Timeout, default value is 600 seconds

Returns:

A pandas DataFrame, Spark DataFrame or pyarrow.Table with the result.

Return type:

DataFrame | Table | DataFrame

Raises:

ValueError – Only direct read eligible queries can be returned as arrow Table.

read_fsql_query_results_arrow(query_id)[source]#

Create a bytes io reader if query returned arrow.

Parameters:

query_id (str)

Return type:

pa.ipc.RecordBatchStreamReader

api_queries_execute(query, branch, dialect='SPARK', timeout=600, **kwargs)[source]#

Queries the foundry sql server.

Parameters:
Return type:

requests.Response

api_queries_status(query_id, **kwargs)[source]#

Get the foundry sql query status.

Parameters:

  • query_id (str) – query id to get status

  • **kwargs – gets passed to APIClient.api_request()

Return type:

requests.Response

api_queries_results(query_id, **kwargs)[source]#

Get query results.

Parameters:

  • query_id (str) – query id to get results

  • **kwargs – gets passed to APIClient.api_request()

Return type:

requests.Response

Contents