Connexion to Terdata are not closed

[KevinMendesCGI]

Hello, I am telling you about a problem encountered on a move2cloud Teradata to GCP project.

Objective of our tests:

Check the validity of data between more than 9000 Teradata and GCP tables.

Desired result of the tool:

The data validation tool, launched individually for each table, must report the results of the comparisons between 2 tables, all over an iteration of approximately 9000 tables. At the end of a comparison, the connection to the database must be cut.

Result obtained:

After several hundred tables compared, the connections not being closed, these connections go into "idle" on Teradata causing a connection overload making it impossible to connect to the database with all the consequences related to this overload

proposition of a solution:

The teradatasql library has a .close() method which should solve this problem.

you could close the connection by destroying the object after sending results. This method would not cause conflict with other client's methods.

On https://github.com/KevinMendesCGI/professional-services-data-validator/blob/ad7f1fcfc10541cf19044a1bf88d16deb1398772/third_party/ibis/ibis_teradata/client.py#L61-L98

class TeradataClient(SQLClient):
    """ An ibis Teradata client implementation """

    query_class = TeradataQuery
    database_class = TeradataDatabase
    table_class = TeradataTable
    dialect = compiler.TeradataDialect

    def __init__(self, host, user_name, password, port=1025, logmech='TD2', use_no_lock_tables=False):
        """Construct a TeradataClient.

        Parameters
        ----------
        host : str
            The Database host to connect to
        user_name : str
            A Database username to connect with
        password : str
            Password for supplied username
        port : Optional[int]
            The database port to connect to (default. 1025)
        logmech : Optional[int]
            The logmech type to connect to (default. TD2)
        """
        self.teradata_config = {
            "host": host,
            "user": user_name,
            "password": password,
            "dbs_port": port,
            "logmech": logmech,
        }

        self.client = teradatasql.connect(**self.teradata_config)
        self.use_no_lock_tables = use_no_lock_tables


    def __del__ (self):
        teradatasql.close()

By using del object the connexion will be close