{"id":1824,"date":"2017-10-30T13:53:38","date_gmt":"2017-10-30T20:53:38","guid":{"rendered":"http:\/\/miriamposner.com\/classes\/dh101f17\/?p=1824"},"modified":"2017-10-30T13:53:38","modified_gmt":"2017-10-30T20:53:38","slug":"using-openrefine-to-clean-classicists-dataset","status":"publish","type":"post","link":"http:\/\/miriamposner.com\/classes\/dh101f17\/2017\/10\/30\/using-openrefine-to-clean-classicists-dataset\/","title":{"rendered":"Using OpenRefine to Clean Classicists Dataset"},"content":{"rendered":"

Our dataset is about American Classicists & Archaeologists. The research questions we have come up with are as follows:<\/span><\/p>\n

    \n
  1. Has the prevalence of female classicists increased over time? <\/span><\/li>\n
  2. Do the regions of granting institutions have any effect on a classicists\u2019 area of study? Has there been a shift in regional activity over time? <\/span><\/li>\n
  3. Is there a trend in areas of study based on classicists\u2019 affiliate institutions?<\/span><\/li>\n
  4. How did the refugee status affect the classicists\u2019 field of study?<\/span><\/li>\n
  5. How did the birthplace and origin of classicists affect their main affiliation and\/or work?<\/span><\/li>\n<\/ol>\n

    For the research questions we are working on above, there are following columns can be manipulated by OpenRefine so that data can be analyzed by other tools better: <\/span><\/p>\n

      \n
    • Area of study & Granting Institution & Main Affiliation & Refuge: Use \u2018Text Facet\u2019 to see if there are terms been grouped separately with the same meaning, and combine them together.<\/span>\n
        \n
      • OpenRefine: <\/span>\n
          \n
        • On the <\/span>Research area & Granting Institution & Main Affiliation & Refuge <\/b>column, click the down arrow, then click <\/span>Edit cells<\/b>, then <\/span>Common transforms<\/b>. Finally, click <\/span>Trim leading and trailing whitespace<\/b>.<\/span><\/li>\n
        • Click on the down-arrow right next to the <\/span>Research area <\/b>column heading. Then select <\/span>Facet<\/b>, and then <\/span>Text Facet<\/b>.<\/span><\/li>\n
        • Click on <\/span>Cluster <\/b>to see if there is any terms have the same meaning, then edit the text to combine them together.<\/span><\/li>\n
        • Double check if there is any left.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n

          \"\" \"\"<\/p>\n

            \n
          • Birthplace: divided into two columns, one with \u2018Birth City\u2019 and one with \u2018Birth State\u2019.<\/span>\n
              \n
            • OpenRefine: <\/span>\n
                \n
              • On the <\/span>Birthplace <\/b>column, click the down arrow, then click <\/span>Edit cells<\/b>, then <\/span>Common transforms<\/b>. Finally, click <\/span>Trim leading and trailing whitespace<\/b>.<\/span><\/li>\n
              • Click the down arrow next to the Birthplace, then <\/span>Edit columns<\/b>, and finally <\/span>Split multi-valued cells<\/b>. Enter a comma and space, since those are the two characters that lie between city and state. Then click <\/span>OK<\/b>.<\/span><\/li>\n
              • Then click on the down arrow, <\/span>Edit column<\/b> and then <\/span>Rename <\/b>two columns.<\/span><\/li>\n
              • Click on the down-arrow right next to the <\/span>Refugee<\/b> column heading. Then select <\/span>Facet<\/b>, and then <\/span>Text Facet<\/b>.<\/span><\/li>\n
              • Click on <\/span>Cluster <\/b>to see if there is any terms have the same meaning, then edit the text to combine them together.<\/span><\/li>\n
              • Double check if there is any left.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n

                \"\" \"\"<\/p>\n

                Moreover, I am interested in how to \u2018sort\u2019 the data and when we gonna to use \u2018transpose\u2019 function. However, the most significant problem with our dataset is there is so many data are missing. We are going to talk with the subject-matter specialist as soon as possible.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

                Our dataset is about American Classicists & Archaeologists. The research questions we have come up with are as follows: Has<\/p>\n","protected":false},"author":164,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_eb_attr":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-1824","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/posts\/1824","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/users\/164"}],"replies":[{"embeddable":true,"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/comments?post=1824"}],"version-history":[{"count":0,"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/posts\/1824\/revisions"}],"wp:attachment":[{"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/media?parent=1824"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/categories?post=1824"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/miriamposner.com\/classes\/dh101f17\/wp-json\/wp\/v2\/tags?post=1824"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}