Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

ConvertFrom-JSON high memory consumption #7698

Copy link
Copy link
@strawgate

Description

@strawgate
Issue body actions

Steps to reproduce

$JSON = @"
{
  "configGlossary:installationAt": "Philadelphia, PA",
  "configGlossary:adminEmail": "ksm@pobox.com",
  "configGlossary:poweredBy": "Cofax",
  "configGlossary:poweredByIcon": "\/images\/cofax.gif",
  "configGlossary:staticPath": "\/content\/static",
  "templateProcessorClass": "org.cofax.WysiwygTemplate",
  "templateLoaderClass": "org.cofax.FilesTemplateLoader",
  "templatePath": "templates",
  "templateOverridePath": "",
  "defaultListTemplate": "listTemplate.htm",
  "defaultFileTemplate": "articleTemplate.htm",
  "useJSP": false,
  "jspListTemplate": "listTemplate.jsp",
  "jspFileTemplate": "articleTemplate.jsp",
  "cachePackageTagsTrack": 200,
  "cachePackageTagsStore": 200,
  "cachePackageTagsRefresh": 60,
  "cacheTemplatesTrack": 100,
  "cacheTemplatesStore": 50,
  "cacheTemplatesRefresh": 15,
  "cachePagesTrack": 200,
  "cachePagesStore": 100,
  "cachePagesRefresh": 10,
  "cachePagesDirtyRead": 10,
  "searchEngineListTemplate": "forSearchEnginesList.htm",
  "searchEngineFileTemplate": "forSearchEngines.htm",
  "searchEngineRobotsDb": "WEB-INF\/robots.db",
  "useDataStore": true,
  "dataStoreClass": "org.cofax.SqlDataStore",
  "redirectionClass": "org.cofax.SqlRedirection",
  "dataStoreName": "cofax",
  "dataStoreDriver": "com.microsoft.jdbc.sqlserver.SQLServerDriver",
  "dataStoreUrl": "jdbc:microsoft:sqlserver:\/\/LOCALHOST:1433;DatabaseName=goon",
  "dataStoreUser": "sa",
  "dataStorePassword": "dataStoreTestQuery",
  "dataStoreTestQuery": "SET NOCOUNT ON;select test='test';",
  "dataStoreLogFile": "\/usr\/local\/tomcat\/logs\/datastore.log",
  "dataStoreInitConns": 10,
  "dataStoreMaxConns": 100,
  "dataStoreConnUsageLimit": 100,
  "dataStoreLogLevel": "debug",
  "maxUrlLength": 500
}
"@

$Count = 39999
$jsonarr = "[" + (($json + ",") * $Count) + $json + "]"
[system.gc]::Collect()
write-host ""
write-host ""
write-host "JSON -- Objects: $($Count + 1)"
write-host "JSON -- Length:  $($jsonarr.length / 1024 / 1024 ) Megabytes"

$BeforeMemory = $((get-process -pid $PID).WorkingSet)
[void] (convertfrom-json $jsonarr)
$AfterMemory = $((get-process -pid $PID).WorkingSet)

$MemoryDiff = $AfterMemory - $BeforeMemory

write-host "Memory Usage -- Before:     $($BeforeMemory  / 1024 / 1024) Megabytes"
write-host "Memory Usage -- After:      $($AfterMemory  / 1024 / 1024) Megabytes"
write-host "Memory Usage -- Difference: $($MemoryDiff  / 1024 / 1024) Megabytes"
write-host "Memory Usage -- Multiple:   $($MemoryDiff / $jsonarr.length)"

output:

JSON -- Objects: 40000
JSON -- Length:  67.4819955825806 Megabytes
Memory Usage -- Before:     210.8359375 Megabytes
Memory Usage -- After:      1240.90625 Megabytes
Memory Usage -- Difference: 1030.0703125 Megabytes
Memory Usage -- Multiple:   15.2643724241892

Actual behavior

Powershell Core ends up with 1 GB of memory used and peaks to 1.6 GB of memory utilization during convertfrom-json

Expected behavior

Powershell Core shouldn't need 1.6 gigabytes of memory to deserialize a 65MB JSON file.

Environment data

Name                           Value
----                           -----
PSVersion                      6.1.0-preview.3
PSEdition                      Core
GitCommitId                    v6.1.0-preview.3
OS                             Microsoft Windows 10.0.17134
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0
Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    Issue-Enhancementthe issue is more of a feature request than a bugthe issue is more of a feature request than a bugResolution-No ActivityIssue has had no activity for 6 months or moreIssue has had no activity for 6 months or moreWG-Cmdlets-Utilitycmdlets in the Microsoft.PowerShell.Utility modulecmdlets in the Microsoft.PowerShell.Utility moduleWG-Engine-Performancecore PowerShell engine, interpreter, and runtime performancecore PowerShell engine, interpreter, and runtime performance

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.