python - How can I force ElementTree to write to disk every change? -

i have deal huge xml file create big database.

from database structure generate xml tree, ends eating memory (over 10gb now), process never end since system can't process new query.

i think solution avoid saving xml structure memory, , dump directly disk whenever add new.

is can do? how?

thanks!

i'm not sure how code works, it's done. depending on how it's structured, create timestamp , compare incoming data old data, or scan file whether current xml has been added. after (assuming new code) following:

path = "path/" name = "filename"  xmlroot = element("root")#create root element xml structure xmlsub = subelement(xmlroot,"sub") subname = subelement(xmlcard,"name") subname.text = "element text"  savename = path + name + ".xml" #constructs location of xml file (path/filename.xml) tree = elementtree(xmlroot) #compiles tree tree.append(savename) #appends specified file 

this output following in document in folder "path" under name "filename.xml"

<root>     <sub>         <name>element text</name>     </sub> </root> 

if wanted scan xml document object, following:

xml = parse("path/filename.xml") namelist = xml.findall("sub/name") #find objects in <name> brackets in namelist:     i.text #convert item in list readable string     #do comparison here 

i hope helps! happy coding!