javascript - Process node growing until eating all memory -


this question same previous one, since became huge repeated edit, prefer ask again properly.

i m working on raspberry pi (raspbian, 512m ram), , doing javascript app in node.js (v0.10.2).

in it, need download video, wich chunk this:

function download (file_url, callback){     var option={host:url.parse(file_url).host, port:80,path:url.parse(file_url).pathname};     var file_name=url.parse(file_url).pathname.split('/').pop();     var file=fs.createwritestream(download_dir+file_name);     http.get(options, function(res){         res.on('data', function(data){             file.write(data);         }).on('end'), function(){             file.end();             callback(download_dir+file_name);         });     }); }; 

wich believe similar download large file node.js avoiding high memory consumption

the problem is, rss of node process don t stop increasing, s start around 100m, , killed os when around 400m (wich maximal ram).

since m new memory leak tracing , running app outside of browser, either can t use or can t understand how use memory tool, ve rss util library.

my question is: why rss keep increasing when m writing chunk of data on fly, and/or search solution?

edit: on second @ link above, noticed use of flags on createwritestream, can t find list nor effect of them, there here know way access such documentation, may related

edit2: m blindly following code of corlosdp, adding flag (appends?) make rss drop of 2m @ random intervals, , make app last longer, still end killed

edit3:i have discovered interesting

i edited download function see in file:

function download (file_url, callback){      var i=0;      var option={host:url.parse(file_url).host, port:80,path:url.parse(file_url).pathname};     var file_name=url.parse(file_url).pathname.split('/').pop();      var file=fs.createwritestream(download_dir+file_name, {flags:'a', encoding:'binary');      setinterval(function(){         console.log(i+"o downloaded");         console.log(util.inspect(process.memoryusage()));     },1000);      http.get(options, function(res){         res.on('data', function(data){              i=i+data.length;              file.write(data);         }).on('end'), function(){             file.end();             callback(download_dir+file_name);         });     }); }; 

and log looked this:

[downloading config file] 0o downloaded 0o downloaded 0o downloaded  [logs of config file getting read, , first file start downloaded]  986o downloaded {rss: ...} 6713830o downloaded {rss:...} 986o downloaded {rss:...} 14421142o downloaded {rss: ...} 986o downloaded {rss:...} 29530702o downloaded {rss:...} ... 

it seems there 2 download in parrallel, keep getting stranger when second file start (in callback of first file download), since there s 3 different download: 986 constant, first file keep appearing, , second file!

doesn t function supposed end when callback reached?

edit5: silly me, forgot clear interval, wich why got this.

i still don t know why code didn t worked, irc user, tried replace

res.on('data', function(data){     ... }); 

by

res.pipe(file); 

it work, , don t go on 60m of ram, while struggling don t go on 400m, don t know why neither he, problem solved.


Comments

Popular posts from this blog

html - How to style widget with post count different than without post count -

How to remove text and logo OR add Overflow on Android ActionBar using AppCompat on API 8? -

javascript - storing input from prompt in array and displaying the array -