Question

I am getting the following error when using table.upsert() to update data.

Exception has occurred: RecursionError
maximum recursion depth exceeded
AttributeError: '_thread._local' object has no attribute 'f_unhandled_exc'

During handling of the above exception, another exception occurred:

File "C:\loader_pyiceberg.py", line 118, in upsert
table.upsert(pyarrow_table,join_cols=join_cols)
File "C:\loader_pyiceberg.py", line 222, in
upsert(iceberg_table,pyarrow,['COL1','COL2','COL3','COL4'])
RecursionError: maximum recursion depth exceeded

This works fine if the no: of rows in pyarrow table (source) is around 400. I am getting the error if the number of rows is around 500. Is there any limitations for this method, based on size of the data?

Could you help me resolve this issue.

Thanks