{"id":1880,"date":"2018-05-12T22:14:56","date_gmt":"2018-05-12T20:14:56","guid":{"rendered":"http:\/\/van-maanen.com\/?p=1880"},"modified":"2018-05-12T22:14:56","modified_gmt":"2018-05-12T20:14:56","slug":"show-content-of-an-avro-file-with-php","status":"publish","type":"post","link":"http:\/\/archief.van-maanen.com\/?p=1880","title":{"rendered":"Show content of an AVRO file with Python"},"content":{"rendered":"<p>This note describes how we can show the content of an AVRO file with Python.<br \/>\nWe use python3 as tool here. We use this from an Anaconda framework. I checked whether this installation already contained an AVRO package, but this wasn&#8217;t the case. Therefore, AVRO was downloaded (as avro-python3-1.8.2.tar.gz) and next command was issued: C:\\ProgramData\\Anaconda3\\python.exe -m pip install C:\\ProgramData\\Anaconda3\\avro-python3-1.8.2.tar.gz. The screen then looked like:<br \/>\n<img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/van-maanen.com\/wp-content\/uploads\/2018\/05\/Anaconda.png\" alt=\"\" width=\"822\" height=\"404\" class=\"alignnone size-full wp-image-1881\" \/>.<br \/>\nOnce, AVRO was installed, an AVRO file can be created with:<\/p>\n<pre>\nimport avro.schema\nfrom avro.datafile import DataFileWriter\nfrom avro.io import DatumWriter\n\nschema = avro.schema.Parse(open('file.avsc', \"r\").read())\n\nwriter = DataFileWriter(open(\"users.avro\", \"wb\"), DatumWriter(), schema)\nwriter.append({\"name\": \"Alyssa\", \"favorite_number\": 256})\nwriter.append({\"name\": \"Ben\", \"favorite_number\": 7, \"favorite_color\": \"red\"})\nwriter.close()\n<\/pre>\n<p>Here use was made of a file that contained the schema:<\/p>\n<pre>\n{\"namespace\": \"example.avro\",\n \"type\": \"record\",\n \"name\": \"User\",\n \"fields\": [\n     {\"name\": \"name\", \"type\": \"string\"},\n     {\"name\": \"favorite_number\",  \"type\": [\"int\", \"null\"]},\n     {\"name\": \"favorite_color\", \"type\": [\"string\", \"null\"]}\n ]\n}\n<\/pre>\n<p>When a file was created, it can be read as:<\/p>\n<pre>\nfrom avro.datafile import DataFileReader\nfrom avro.io import DatumReader\n\nreader = DataFileReader(open(\"users.avro\", \"rb\"), DatumReader())\nfor user in reader:\n    print (user)\nreader.close()\n<\/pre>\n<p>This latter programme is nice: another avro file can be read. I downloaded another avro file (called twitter.avro) and the corresponding outcome looked like:<\/p>\n<pre>\n\nrunfile('C:\/Users\/tmaanen\/.spyder-py3\/avroTomLees.py', wdir='C:\/Users\/tmaanen\/.spyder-py3')\n{'username': 'miguno', 'tweet': 'Rock: Nerf paper, scissors is fine.', 'timestamp': 1366150681}\n{'username': 'BlizzardCS', 'tweet': 'Works as intended.  Terran is IMBA.', 'timestamp': 1366154481}\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>This note describes how we can show the content of an AVRO file with Python. We use python3 as tool here. We use this from an Anaconda framework. I checked whether this installation already contained an AVRO package, but this wasn&#8217;t the case. Therefore, AVRO was downloaded (as avro-python3-1.8.2.tar.gz) and next command was issued: C:\\ProgramData\\Anaconda3\\python.exe [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-1880","post","type-post","status-publish","format-standard","hentry","category-allgemein"],"_links":{"self":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/posts\/1880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1880"}],"version-history":[{"count":0,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/posts\/1880\/revisions"}],"wp:attachment":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1880"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}