Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

Micah-Kolide
Copy link
Contributor

Relates to #7914

I narrowed down the/a memory leak in unified_log caused from the OSLogEnumerator. Unless all objects are enumerated, they are held in memory.

Example:

Pre PR:

On startup memory consumption looks like this:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 63022 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
 25936 409002864  0.1 63022 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

After running a very heavy unified_log query such as:

select count(*) from unified_log where timestamp >= (select unix_time-100000 from time);

The memory jumps and doesn't release after the query is finished:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 63022 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
559920 410133152  1.7 63022 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

Post PR:

On startup memory consumption looks like this:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 50377 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
 25392 408852336  0.1 50377 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

After running a very heavy unified_log query such as:

select count(*) from unified_log where timestamp >= (select unix_time-100000 from time);

The memory jumps but releases as each OSLogEntryLog is iterated over until the query is finished:

Micah@Macbook-Pro ~/osquery/build (master) $ ps x -p 50377 -o rss,vsz,%mem,pid,command,args | sort -nr | grep -i osquery | grep -v 'grep'
195808 410110032  0.6 50377 /Users/micahsore /Users/micahsorenson/osquery/build/osquery/osqueryd -S

An unfortunate con to this is of course the increase in runtime while waiting for the iterator to finish.

@Micah-Kolide Micah-Kolide requested review from a team as code owners February 16, 2024 06:35
Copy link
Member

@zwass zwass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems suspicious but possibly a bug in macOS? What I'd rather see is that the autoreleasepool handles this (but I guess it's not?) or that there is a function that needs to be called to release these resources. I couldn't find any indication of such a function being available when I searched the docs.

@directionless directionless merged commit 025ac34 into osquery:master Feb 27, 2024
@directionless
Copy link
Member

https://eclecticlight.co/2024/07/19/writing-a-third-generation-log-browser-using-swiftui-3-lost-memory/ has a similar conclusion -- that there is a memory leak.

@directionless
Copy link
Member

The oslog enumerator conforms to https://developer.apple.com/documentation/foundation/nsfastenumeration and I wonder if its countByEnumerating (which has a len parameter) would solve this memory leak. Depends a bit on where in the macOS code it is

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.