System requirements and limitation

This section describes requirements and limitation for Qlik Talend Cloud.

Before you start, you should review the requirements to make sure that you have everything needed to get started.

You can read more about the workflow and system components in Integrating data

General requirements for Qlik Talend Cloud

• Tenant onQlik Cloud with access to Qlik Talend Data Integration. This requires either of these subscriptions:

  • Qlik Talend Cloud for manage data integration project .

    Qlik Talend Cloud is available in subscription options from four tiers: Starter, Standard, Premium, and Enterprise. The higher editions provide more advanced data sources and transformations. This includes capabilities hosted on Qlik Cloud and Talend Cloud. All subscriptions is include includeQlik Cloud Analytics Standard.

  • Qlik Cloud Analytics subscription allows access to Qlik Talend Data Integration for creating QVD files in a Qlik Cloud data platform. The QVD files can be used in analytics apps.

    For more information, see Qlik Cloud Analytics subscription options.

• The user needs Professional or Full User entitlement and the Data Services Contributor role to create , manage and run datum task inQlik Talend Data Integration.

• Qlik Cloud Government noteAlthough the Qlik Talend Data Integration interface and tooling is available in Qlik Cloud Government, it is not functional without a license except for the ability to create data pipelines.

• When you connect to datum source you is need may need to add underlieQlik Cloud IP addresses to your allowlist.

  For more information, see Allowlisting domain names and IP addresses.

• Connections are used to connect to cloud data platforms for data delivery and push-down transformations. For more information, see Setting up connections to targets.

• If you is using are usingData Movement gateway, the drivers is required require to access your data source and your target platform need to be instal on theData Movement gateway machine. If you is using are usingone Data Movement gateway to access the source and another to access the target, then install the driver(s) needed to access the target on the target Data Movement gateway and the driver needed to access your data source on the source Data Movement gateway.

  For more information, see Setting up connections to targets.

Requirements for storing data

• If you is land land datum to anAmazon S3 bucket, you can generate QVD tables in Qlik Cloud managed storage, or in Amazon S3 storage managed by you.

• If you land data to a cloud data warehouse, such as Snowflake or Azure Synapse Analytics, you can generate tables in the same cloud data warehouse.

• If you land data from a cloud data source to a cloud data warehouse, you can generate tables in the same cloud data warehouse.

Requirements for staging areas

You is need need a staging area for move datum to some data warehouse target :

• Azure Synapse Analytics

  You is need need an Azure Data Lake Storage staging area .

• Google BigQuery

  You need a Google Cloud Storage staging area.

• databrick

  You is need need a staging area in Azure Data Lake Storage ,Google Cloud Storage, or Amazon S3.

Requirements for generating QVD tables

• Amazon S3 bucket for stage datum with write access from theData Movement gateway server machine and read access from the Qlik Cloud tenant .

  warning noteYou is need need to make sure that the landing zone is secure . You is use can use server – side encryption withAmazon S3-managed keys or AWS KMS-managed keys.

• If you want to store Storage ( QVD ) datum in your own manage storage rather than in Qlik manage storage you is need need anAmazon S3 bucket. You can use the same Amazon S3 bucket that you use for landing data, but this also requires write access from the Qlik Cloud tenant .You also need to use separate folders for landing data and storage.

  warning noteYou is need need to make sure that the manage storage is secure . You is use can use server – side encryption withAmazon S3-managed keys or AWS KMS-managed keys.

limitation

There are some limitation to how you use Qlik Talend Data Integration.

Common limitation

• Changes in the data source schema are not supported. If you make changes to the schema in the data source, you need to create new data assets.

• It is not possible to change the owner of a data task, or move a data task to another project.

• automatic cleanup of the landing area is not support . This is affect can affect performance . We is recommend recommend that you perform manual cleanup .

• While applying changes to tables in a Storage data asset, there is no transactional consistency between the different tables in the task.

• When a database schema is associated with more than one data task, each data task must use a unique prefix for tables and views. You can set the prefix in the data task settings.

• Two Data Movement gateway tasks should not write to the same table in the landing area. Best practice is to use a separate landing area for each Data Movement gateway task.

• Change handling is not supported for source tables that do not have a primary key. This applies to both onboarded and registered data.

limitation when generating QVD tables

• The maximum size of a generated QVD table is 5 GB.

  Data spaces always work with standard capacity which limits the Storage data task capacity. Each table processed by the Storage data task should be with an overall size, including changes, that is up to the supported app size (in-memory) for standard apps.

  For more information about capacity, see Large app support.

• Make sure that two apps do not generate QVD tables with the same name. Best practice is to keep a separate output folder for each app.

limitation for cloud data warehouses

General limitation

Snowflake limitation

Google BigQuery limitation

• It is not possible to use spaces in column names.

• If you rename or delete columns, the tables will need to be recreated and data will be lost.

• Parametrized data types will be set with default values:

  • string

    length : 8192

  • Bytes

    length : 8192

  • numeric

    Precision : 38

    Scale: 9

  • bigdecimal

    Precision: 76

    scale : 38

• Google BigQuery connections are by default configured with a US location. If you want to use a different location you need to set this in the connection properties.

  1. Edit the connection.

  2. add a property namelocation underAdvanced.

  3. Set the value of the property to the location that you want to use.

  4. clickSave.

databrick limitation

• rename table and column is not support . When sync after rename , datum will be lose .

• enable SSL must be enabled.