{"id":1898,"date":"2018-05-21T22:14:26","date_gmt":"2018-05-21T20:14:26","guid":{"rendered":"http:\/\/van-maanen.com\/?p=1898"},"modified":"2018-05-21T22:14:26","modified_gmt":"2018-05-21T20:14:26","slug":"copying-content-of-an-oracle-table-into-an-avro-file","status":"publish","type":"post","link":"http:\/\/archief.van-maanen.com\/?p=1898","title":{"rendered":"Copying content of an Oracle table into an avro file"},"content":{"rendered":"<p>Below, you will find a listing on how to copy the content of an Oracle table into an avro file. The trick is quite straight forward. A table is read via a cursor. Each record is then appended to an avro file. The scheme is:<\/p>\n<pre>\n{\n\n  \"namespace\": \"example.avro\",\n  \"type\": \"record\",\n  \"name\": \"User\",\n  \"fields\": [\n      {\"name\": \"name\", \"type\": \"string\"},\n      {\"name\": \"number\",  \"type\": [\"int\", \"null\"]},\n      {\"name\": \"local\", \"type\": [\"string\", \"null\"]}\n  ]\n}\n<\/pre>\n<p>So:<\/p>\n<pre>\nimport avro.schema\nfrom avro.datafile import DataFileWriter\nfrom avro.io import DatumWriter\nimport cx_Oracle\nteller =0\n\nschema = avro.schema.Parse(open(\"user.avsc\").read())\nwriter = DataFileWriter(open(\"users.avro\", \"wb\"), DatumWriter(), schema)\ncon = cx_Oracle.connect('C##HR\/xxx12345@van-maanen.com\/orcl')\n\ncur = con.cursor()\ncur.execute('select name, local from countries')\nfor result in cur:\n    teller = teller + 1\n    writer.append({\"name\": result[0], \"number\": teller, \"local\": result[1]})\n    \ncur.close()\ncon.close()\nwriter.close()\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Below, you will find a listing on how to copy the content of an Oracle table into an avro file. The trick is quite straight forward. A table is read via a cursor. Each record is then appended to an avro file. The scheme is: { &#8220;namespace&#8221;: &#8220;example.avro&#8221;, &#8220;type&#8221;: &#8220;record&#8221;, &#8220;name&#8221;: &#8220;User&#8221;, &#8220;fields&#8221;: [ {&#8220;name&#8221;: [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-1898","post","type-post","status-publish","format-standard","hentry","category-allgemein"],"_links":{"self":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/posts\/1898","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1898"}],"version-history":[{"count":0,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=\/wp\/v2\/posts\/1898\/revisions"}],"wp:attachment":[{"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1898"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1898"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/archief.van-maanen.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1898"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}