xmltodict，xml与json的相互转换

源码：

在开发中经常遇到string、xml、json、dict对象的相互转换，这个工具和这里的方法全部都能够搞定。

XML文件转换流程

注意：以下代码只是示范逻辑，不能直接运行。

import osimport timeimport lxmlfrom lxml import etreeimport xmltodict, sys, gc# 递归解析xml文件context = etree.iterparse(osmfile,tag=["node","way","relation"])fast_iter(context, process_element, maxline)...# xml对象转为字符串elem_data = etree.tostring(elem)# 生成dict对象elem_dict = xmltodict.parse(elem_data)# 从dict产生json字符串elem_jsonStr = json.dumps(elem_dict)# 从json字符串产生json对象json_obj = json.dumps(elem_jsonStr)

递归解析XML

etree递归读取xml结构数据（占用资源少）:

XML字符串转为json对象支持库 :  

xmltodict.parse()会将字段名输出添加@和#，在Spark查询中会引起问题，需要去掉。如下设置即可：

xmltodict.parse(elem_data,attr_prefix="",cdata_key="")

编码和错误xml文件恢复

如下：

magical_parser = lxml.etree.XMLParser(encoding='utf-8', recover=True)  tree = etree.parse(StringIO(your_xml_string), magical_parser) #or pass in an open file object

先将element转为string，然后生成dict，再用json.dump()产生json字符串。

elem_data = etree.tostring(elem)elem_dict = xmltodict.parse(elem_data)elem_jsonStr = json.dumps(elem_dict)

可以使用json.loads(elem_jsonStr)创建出可编程的json对象。

xmltodict的用法

xmltodict is a Python module that makes working with XML feel like you are working with , as in this :

>>> print(json.dumps(xmltodict.parse("""...  
    
     ...    
     
      ...      
      
       elements
      ...      
      
       more elements
      ...    
     ...    
     
      ...      element as well...    ...  ...  """), indent=4)){ "mydocument": { "@has": "an attribute",        "and":         {          "many": ["elements",  "more elements"]        },         "plus": {"@a": "complex", "#text": "element as well"        }    }}
     
    

Namespace support

By default, xmltodict does no XML namespace processing (it just treats namespace declarations as regular node attributes), but passing process_namespaces=True will make it expand namespaces for you:

>>> xml = """... 
    
     
      1
     ...   
     
      2
     ...   
     
      3
     ... ... """>>> xmltodict.parse(xml, process_namespaces=True) == {...     'http://defaultns.com/:root': {...         'http://defaultns.com/:x': '1',...         'http://a.com/:y': '2',...         'http://b.com/:z': '3',...     }... }True
    

It also lets you collapse certain namespaces to shorthand prefixes, or skip them altogether:

>>> namespaces = {...     'http://defaultns.com/': None, # skip this namespace...     'http://a.com/': 'ns_a', # collapse "http://a.com/" -> "ns_a"... }>>> xmltodict.parse(xml, process_namespaces=True, namespaces=namespaces) == {...     'root': {...         'x': '1',...         'ns_a:y': '2',...         'http://b.com/:z': '3',...     },... }True

Streaming mode

xmltodict is very fast (-based) and has a streaming mode with a small memory footprint, suitable for big XML dumps like or :

>>> def handle_artist(_, artist):...     print artist['name']...     return True>>> >>> xmltodict.parse(GzipFile('discogs_artists.xml.gz'),...     item_depth=2, item_callback=handle_artist)A Perfect CircleFantômasKing CrimsonChris Potter...

It can also be used from the command line to pipe objects to a script like this:

import sys, marshalwhile True:    _, article = marshal.load(sys.stdin)    print article['title']

$ cat enwiki-pages-articles.xml.bz2 | bunzip2 | xmltodict.py 2 | myscript.pyAccessibleComputingAnarchismAfghanistanHistoryAfghanistanGeographyAfghanistanPeopleAfghanistanCommunicationsAutism...

Or just cache the dicts so you don't have to parse that big XML file again. You do this only once:

$ cat enwiki-pages-articles.xml.bz2 | bunzip2 | xmltodict.py 2 | gzip > enwiki.dicts.gz

And you reuse the dicts with every script that needs them:

$ cat enwiki.dicts.gz | gunzip | script1.py$ cat enwiki.dicts.gz | gunzip | script2.py...

Roundtripping

You can also convert in the other direction, using the unparse() method:

>>> mydict = {...     'response': {...             'status': 'good',...             'last_updated': '2014-02-16T23:10:12Z',...     }... }>>> print unparse(mydict, pretty=True)
    
    
         
     
      good
         
     
      2014-02-16T23:10:12Z
     
    

Text values for nodes can be specified with the cdata_key key in the python dict, while node properties can be specified with the attr_prefix prefixed to the key name in the python dict. The default value for attr_prefix is @ and the default value for cdata_key is #text.

>>> import xmltodict>>> >>> mydict = {...     'text': {...         '@color':'red',...         '@stroke':'2',...         '#text':'This is a test'...     }... }>>> print xmltodict.unparse(mydict, pretty=True)
    
    
     This is a test
    

Ok, how do I get it?

Using pypi

You just need to:

$ pip install xmltodict

RPM-based distro (Fedora, RHEL, …)

There is an .

$ sudo yum install python-xmltodict

Arch Linux

There is an .

$ sudo pacman -S python-xmltodict

Debian-based distro (Debian, Ubuntu, …)

There is an .

$ sudo apt install python-xmltodict