fix freezing with deepcopy model#3083
Open
SergeevVladislav wants to merge 37 commits intocatboost:mastercatboost/catboost:masterfrom
SergeevVladislav:deepcopy-model-freezeSergeevVladislav/catboost:deepcopy-model-freezeCopy head branch name to clipboard
Open
fix freezing with deepcopy model#3083SergeevVladislav wants to merge 37 commits intocatboost:mastercatboost/catboost:masterfrom SergeevVladislav:deepcopy-model-freezeSergeevVladislav/catboost:deepcopy-model-freezeCopy head branch name to clipboard
SergeevVladislav wants to merge 37 commits intocatboost:mastercatboost/catboost:masterfrom
SergeevVladislav:deepcopy-model-freezeSergeevVladislav/catboost:deepcopy-model-freezeCopy head branch name to clipboard
Conversation
andrey-khropov
requested changes
Apr 29, 2026
| # when model_blob is cleared during subsequent fit() calls. | ||
| # See https://github.com/catboost/catboost/issues/2905 | ||
| cdef EModelType modelType = string_to_model_type('cbm') | ||
| cdef TFullModel tmp_model = ReadModel(<char*>serialized_model_str, len(serialized_model_str), modelType) |
Member
There was a problem hiding this comment.
Replacing ReadZeroCopyModel with ReadModel is ineffective - additional data structures that aren't needed when we use ReadZeroCopyModel will be allocated and filled, it means an additional copy of the model data will consume RAM, even if only temporarily.
I suggest to fix the logic around model_blob instead. It must always be cleared only after the corresponding TFullModel object in self.__model is destroyed.
commit_hash:c975063237fb34a806e97350909d01bafb549bd9
commit_hash:1bc6c828ee9df6f8b0b87cc8a6bbd3c5e552fc37
commit_hash:a2da9a8efcf62169b36a6c950e0a7d4550576eb6
commit_hash:ceb446c9a1778ec61762d5addeac8d658c8ccd29
commit_hash:0912d7a08b0b2ae26b6298108a9fdd57cabbc77a
commit_hash:3dc71807ca8b74fbc84807633ded4fca86b0151e
commit_hash:88bc9543595f9b630942d63aa1c3aee154056e92
commit_hash:a4449ed4850ea3acd49ac3b7eb51fd5f04905285
commit_hash:aed803165529b8acc6d55d3404565df5ea49bbdd
commit_hash:a2e646e7dba9d5ab976b4640e870475b25ca47a5
…t; also fix some releaved issues commit_hash:dd15e713a6a83c8a14f1df2f011fa06f189f4a00
commit_hash:c574e7092eae2a9ed715448139ca905f0af5562c
commit_hash:9a5da6e51df2e59de0c2558361b5026e3651bfaa
commit_hash:facd5ff655d7cbe450ca8081220e46ef9930f39b
commit_hash:84610a30b17de408952f5c68dbc32b18b753b247
commit_hash:cbe89a375a914ceb360be4d0e530ea15a09972f5
Описание проблемы в тикете commit_hash:9738f25866ad42e1e6f24191fdbc64feac814c2b
commit_hash:4e10c7121ae27e02301dd9dc617ccc770219fbaf
commit_hash:023e252f0989086f07ccde5133ebbe42d3e9652e
commit_hash:d357056169dd75e367dae4eaa2e392e273ea44ce
commit_hash:16bdb9692ce7f99033b748ff6687507315b99f4e
commit_hash:4a07e32440938bd609d8fff5681f0c4dd1627e5f
commit_hash:2d2808f61599fcfea314ad660585e984d50ffbb3
commit_hash:437cb8244d02857a8ae738d35e3e4b99dd1e41b3
Fixes a crash in default loss inference when training CatBoostClassifier with 2D labels. The update safely handles (n, 1) targets and adds default loss inference for true multilabel (n, k) targets, selecting `MultiLogloss` for binary labels and `MultiCrossEntropy` for soft labels. Includes regression tests. Fixes catboost#3020 --- Pull Request resolved: catboost#3037 commit_hash:51ba22b1b7dab4d146be982597989122a8091a00
commit_hash:38a6bb13e794b4589027ac8c9575c17e889dcbf3
commit_hash:37d62f8847b0d16829524fe0b8dedaacebd06072
commit_hash:31e73ba9a5fc6d95af61246f5e54fa7fc9f340a2
commit_hash:3a1dd6d39ef1e6ac222004f2d6575ef82416dc3b
#| || **<a href="https://nda.ya.ru/t/Q-paCaMv7atRRN" target="_blank"> Echo tests</a>** || |# commit_hash:be0273bf2a568ed2e290a87e392a85a6b4e7eb84
commit_hash:e69b77c6643e19c5d9e0fbd4a14ceccecb60a892
commit_hash:6b7a1917a8bbf484b263ca20de69cb26b5923c45
commit_hash:1b41e9fd0b9319d768536a37c6d9b7398e5e3342
commit_hash:e3c4eb8b9e579225caf5360daefcb2890ac9dbf2
commit_hash:980d4cbbf7e52612e834f92333fb2932422cdf3f
…l destruction ReadZeroCopyModel is restored in _deserialize_model to avoid an extra copy of model data. model_blob is now cleared only after the TFullModel object that references it has been destroyed. This addresses the review comment in PR catboost#3083. Fixes catboost#2905
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#2905