Difference between revisions of "Help:Import Spreadsheet"

From DAS Wiki
Line 2: Line 2:
  
 
==CSV Files==
 
==CSV Files==
In general, the process requires that the spreadsheet be exported to something called a comma-separated-values file (or CSV, for short). Each row in that file will correspond to a future page on the wiki, and will be populated with whatever fields of data you have that correspond to the wiki fields. Nearly all spreadsheet programs have an "export to CSV" option, so the export to that format should be easy.
+
In general, the process requires that the spreadsheet be exported to something called a comma-separated-values file (or CSV, for short) as an interim step, which file can then be imported into the wiki. Each row in that file will correspond to a future page on the wiki, and will be populated with whatever fields of data you have that correspond to the wiki fields. Nearly all spreadsheet programs have an "export to CSV" option, so the export to that format should be easy.
  
However, the file should be prepared in a particular format prior to export. Here are general considerations:
+
However, the file must be prepared in a particular format prior to export. Here are general considerations:
  
* Any fields that contain geographic coordinates (lat/long) must be decimal-degree values, and in a string " " separated by a comma, i.e "39.2356,-74.2356" otherwise they'll import, but won't be the right location. Note that decimal degrees is not the same as degrees, minutes and seconds. See the difference: 30.5 degrees (which is decimal notation) is the equivalent of 30°30’00” (degrees, minutes, seconds notation). Do not take something looking like 30°23’46” and enter it in as 30.2346. They are not equivalent and you'll find the location will not be where you intend.
+
* The header record (field names, etc) which is the first line of the CSV file has a particular format requirement. The page field ''must'' be named Title and the rest of the fields ''must'' be named Table_Name[field_name] (Table_name to be substituted for whatever table the spreadsheet is to be imported into. Currently we have Arch, buildings, Firms and Grounds.)
  
* Any commas in any of the fields must be within quotes. For instance, brick, masonry, stone MUST be "brick, masonry, stone" or the import will fail.
+
* Any fields that contain geographic coordinates (lat/long) must be decimal-degree values, and in a string " " separated by a comma, i.e "39.2356,-74.2356" otherwise they'll import, but won't be the right location. Note that decimal degrees notation is not the same as "degrees, minutes and seconds" notation. For example, 30.5 degrees (which is decimal degrees notation) is the equivalent of 30°30’00” (degrees, minutes, seconds notation). Do not take something looking like 30°23’46” and enter it in as 30.2346. They are not equivalent and you'll find the location will not be where you intend. If you have degrees, minutes and seconds, use [https://www.fcc.gov/media/radio/dms-decimal this conversion helper].
 +
 
 +
* Any commas in any of the fields must be within quotes. For instance, the entry brick, masonry, stone ''must'' be changed to "brick, masonry, stone" or the import will fail.
  
 
* Make sure there isn't already a page for the record in the wiki. If there is, delete that record from the CSV and enter its data into the wiki by hand.
 
* Make sure there isn't already a page for the record in the wiki. If there is, delete that record from the CSV and enter its data into the wiki by hand.
 
* The header record (field names, etc) which is the first line of the CSV file has a particular format requirement. The page field MUST be named Title and the rest of the fields MUST be named Table_Name[field_name] (Table_name to be substituted for whatever table the spreadsheet is to be imported into. Currently we have Arch, buildings, Firms and Grounds.)
 
  
 
==The Table Formats==
 
==The Table Formats==
 
Note that you do not have to have an input for every field. You might not have any entries in ''any'' of the records for a particular field. However, the fields that you do have must be named (in the top row of the CSV file) ''exactly'' the field names below, including underscores, if any. And the names are case sensitive.
 
Note that you do not have to have an input for every field. You might not have any entries in ''any'' of the records for a particular field. However, the fields that you do have must be named (in the top row of the CSV file) ''exactly'' the field names below, including underscores, if any. And the names are case sensitive.
 +
 +
Fields of type "File" hold a pointer to an image on the wiki. As the import process likely takes place prior to the upload of any associated images, we recommend leaving this type field blank during the import, and then returning later to upload the image and update the wiki page.
 +
 
===The Arch Table===
 
===The Arch Table===
 
{|class="wikitable"
 
{|class="wikitable"
Line 24: Line 27:
 
|'''portrait'''
 
|'''portrait'''
 
|File
 
|File
|
+
|We recommend leaving this field blank here. See above.
 
|-
 
|-
 
|'''birth_date'''
 
|'''birth_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''birth_approx'''
 
|'''birth_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''birth_place'''
 
|'''birth_place'''
Line 40: Line 43:
 
|'''death_date'''
 
|'''death_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''death_approx'''
 
|'''death_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''death_place'''
 
|'''death_place'''
Line 60: Line 63:
 
|'''associated_firms'''
 
|'''associated_firms'''
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|'''burial_place'''
 
|'''burial_place'''
Line 72: Line 75:
 
|'''signature'''
 
|'''signature'''
 
|File
 
|File
|
+
|We recommend leaving this field blank here. See above.
 
|-
 
|-
 
|'''awards'''
 
|'''awards'''
Line 89: Line 92:
 
|'''image'''
 
|'''image'''
 
|File
 
|File
|
+
|We recommend leaving this field blank here. See above.
 
|-
 
|-
 
|'''address'''
 
|'''address'''
Line 97: Line 100:
 
|'''Geo'''
 
|'''Geo'''
 
|Coordinates
 
|Coordinates
|
+
|See instructions above for formatting.
 
|-
 
|-
 
|'''altitude'''
 
|'''altitude'''
Line 109: Line 112:
 
|'''building_type'''
 
|'''building_type'''
 
|String
 
|String
|
+
|This will govern whether the entry appears on maps or not. See instructions.
 
|-
 
|-
 
|'''architectural_style'''
 
|'''architectural_style'''
Line 117: Line 120:
 
|'''height'''
 
|'''height'''
 
|Float
 
|Float
|
+
|Enter number (feet) only.
 
|-
 
|-
 
|'''material'''
 
|'''material'''
Line 125: Line 128:
 
|'''floor_count'''
 
|'''floor_count'''
 
|Float
 
|Float
|
+
|Enter number only.
 
|-
 
|-
 
|'''floor_area'''
 
|'''floor_area'''
 
|Float
 
|Float
|
+
|Enter number (square feet) only.
 
|-
 
|-
 
|'''elevator_count'''
 
|'''elevator_count'''
 
|Integer
 
|Integer
|
+
|Enter number only.
 
|-
 
|-
 
|'''structural_system'''
 
|'''structural_system'''
Line 173: Line 176:
 
|'''contractors'''
 
|'''contractors'''
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|'''awards'''
 
|'''awards'''
Line 181: Line 184:
 
|'''groundbreaking_date'''
 
|'''groundbreaking_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''groundbreaking_date_approx'''
 
|'''groundbreaking_date_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''start_date'''
 
|'''start_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''start_date_approx'''
 
|'''start_date_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''topped_out_date'''
 
|'''topped_out_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''topped_out_date_approx'''
 
|'''topped_out_date_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''completion_date'''
 
|'''completion_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''completion_date_approx'''
 
|'''completion_date_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''opened_date'''
 
|'''opened_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''opened_date_approx'''
 
|'''opened_date_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''cost'''
 
|'''cost'''
 
|Float
 
|Float
|
+
|Enter number (dollars) only.
 
|-
 
|-
 
|'''cost_approx'''
 
|'''cost_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''renovation_date'''
 
|'''renovation_date'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''renovation_date_approx'''
 
|'''renovation_date_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''ren_architect'''
 
|'''ren_architect'''
Line 265: Line 268:
 
|'''ren_contractors'''
 
|'''ren_contractors'''
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|'''ren_cost'''
 
|'''ren_cost'''
 
|Float
 
|Float
|
+
|Enter number (dollars) only.
 
|-
 
|-
 
|'''ren_cost_approx'''
 
|'''ren_cost_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''ren_awards'''
 
|'''ren_awards'''
Line 281: Line 284:
 
|'''demolished'''
 
|'''demolished'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''demolished_approx'''
 
|'''demolished_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''national_register_id'''
 
|'''national_register_id'''
Line 303: Line 306:
 
|logo
 
|logo
 
|File
 
|File
|
+
|We recommend leaving this field blank here. See above.
 
|-
 
|-
 
|image
 
|image
 
|File
 
|File
|
+
|We recommend leaving this field blank here. See above.
 
|-
 
|-
 
|firm_type
 
|firm_type
 
|List of String
 
|List of String
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|founders
 
|founders
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|antecedents
 
|antecedents
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|successors
 
|successors
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|personnel
 
|personnel
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|location
 
|location
Line 335: Line 338:
 
|founded
 
|founded
 
|Start date
 
|Start date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|founded_approx
 
|founded_approx
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|dissolved
 
|dissolved
 
|End date
 
|End date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|dissolved_approx
 
|dissolved_approx
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|awards
 
|awards
 
|List of String
 
|List of String
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|significant_projects
 
|significant_projects
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|website
 
|website
Line 373: Line 376:
 
|'''image'''
 
|'''image'''
 
|File
 
|File
|
+
|We recommend leaving this field blank here. See above.
 
|-
 
|-
 
|'''address'''
 
|'''address'''
Line 381: Line 384:
 
|'''perimeter'''
 
|'''perimeter'''
 
|Text
 
|Text
|
+
|See instructions above for formatting.
 
|-
 
|-
 
|'''Geo'''
 
|'''Geo'''
 
|Coordinates
 
|Coordinates
|
+
|See instructions above for formatting.
 
|-
 
|-
 
|'''founder'''
 
|'''founder'''
 
|List of String
 
|List of String
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|'''owner'''
 
|'''owner'''
Line 397: Line 400:
 
|'''builder'''
 
|'''builder'''
 
|List of String
 
|List of String
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|'''landscape_architect'''
 
|'''landscape_architect'''
Line 405: Line 408:
 
|'''other_designers'''
 
|'''other_designers'''
 
|List of Page
 
|List of Page
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|'''acreage'''
 
|'''acreage'''
 
|Float
 
|Float
|
+
|Bare acreage amount here, with no text.
 
|-
 
|-
 
|'''elevation'''
 
|'''elevation'''
Line 417: Line 420:
 
|'''founded'''
 
|'''founded'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''founded_approx'''
 
|'''founded_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''opened'''
 
|'''opened'''
 
|Date
 
|Date
|
+
|mm/dd/yyyy OR mm/yyyy OR yyyy.
 
|-
 
|-
 
|'''opened_approx'''
 
|'''opened_approx'''
 
|Boolean
 
|Boolean
|
+
|Enter 1 for Yes, leave blank for No.
 
|-
 
|-
 
|'''type'''
 
|'''type'''
 
|List of String
 
|List of String
|
+
|See explanation of "lists" above.
 
|-
 
|-
 
|'''NRHP_ID'''
 
|'''NRHP_ID'''

Revision as of 17:05, July 1, 2021

We know that many researchers have used spreadsheets to organize their research, and it would be a waste of time if, in bringing that data into the wiki, it had to be entered all in by hand again. Fortunately the wiki has an import function where properly-formatted spreadsheet data can be automatically imported into the wiki.

CSV Files

In general, the process requires that the spreadsheet be exported to something called a comma-separated-values file (or CSV, for short) as an interim step, which file can then be imported into the wiki. Each row in that file will correspond to a future page on the wiki, and will be populated with whatever fields of data you have that correspond to the wiki fields. Nearly all spreadsheet programs have an "export to CSV" option, so the export to that format should be easy.

However, the file must be prepared in a particular format prior to export. Here are general considerations:

  • The header record (field names, etc) which is the first line of the CSV file has a particular format requirement. The page field must be named Title and the rest of the fields must be named Table_Name[field_name] (Table_name to be substituted for whatever table the spreadsheet is to be imported into. Currently we have Arch, buildings, Firms and Grounds.)
  • Any fields that contain geographic coordinates (lat/long) must be decimal-degree values, and in a string " " separated by a comma, i.e "39.2356,-74.2356" otherwise they'll import, but won't be the right location. Note that decimal degrees notation is not the same as "degrees, minutes and seconds" notation. For example, 30.5 degrees (which is decimal degrees notation) is the equivalent of 30°30’00” (degrees, minutes, seconds notation). Do not take something looking like 30°23’46” and enter it in as 30.2346. They are not equivalent and you'll find the location will not be where you intend. If you have degrees, minutes and seconds, use this conversion helper.
  • Any commas in any of the fields must be within quotes. For instance, the entry brick, masonry, stone must be changed to "brick, masonry, stone" or the import will fail.
  • Make sure there isn't already a page for the record in the wiki. If there is, delete that record from the CSV and enter its data into the wiki by hand.

The Table Formats

Note that you do not have to have an input for every field. You might not have any entries in any of the records for a particular field. However, the fields that you do have must be named (in the top row of the CSV file) exactly the field names below, including underscores, if any. And the names are case sensitive.

Fields of type "File" hold a pointer to an image on the wiki. As the import process likely takes place prior to the upload of any associated images, we recommend leaving this type field blank during the import, and then returning later to upload the image and update the wiki page.

The Arch Table

Field Name Type Comments
portrait File We recommend leaving this field blank here. See above.
birth_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
birth_approx Boolean Enter 1 for Yes, leave blank for No.
birth_place String
death_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
death_approx Boolean Enter 1 for Yes, leave blank for No.
death_place String
nationality String
alma_mater String
associated_firms List of Page See explanation of "lists" above.
burial_place String
spouse String
signature File We recommend leaving this field blank here. See above.
awards String
significant_design String

The buildings Table

Field Name Type Comments
image File We recommend leaving this field blank here. See above.
address String
Geo Coordinates See instructions above for formatting.
altitude String
owner String
building_type String This will govern whether the entry appears on maps or not. See instructions.
architectural_style String
height Float Enter number (feet) only.
material String
floor_count Float Enter number only.
floor_area Float Enter number (square feet) only.
elevator_count Integer Enter number only.
structural_system String
architect Page
architecture_firm Page
structural_engineer String
civil_engineer String
m_e_engineer String
other_designers String
surveyor String
landscape_architect String
contractors List of Page See explanation of "lists" above.
awards String
groundbreaking_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
groundbreaking_date_approx Boolean Enter 1 for Yes, leave blank for No.
start_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
start_date_approx Boolean Enter 1 for Yes, leave blank for No.
topped_out_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
topped_out_date_approx Boolean Enter 1 for Yes, leave blank for No.
completion_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
completion_date_approx Boolean Enter 1 for Yes, leave blank for No.
opened_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
opened_date_approx Boolean Enter 1 for Yes, leave blank for No.
cost Float Enter number (dollars) only.
cost_approx Boolean Enter 1 for Yes, leave blank for No.
renovation_date Date mm/dd/yyyy OR mm/yyyy OR yyyy.
renovation_date_approx Boolean Enter 1 for Yes, leave blank for No.
ren_architect String
ren_firm String
ren_l_a String
ren_str_engineer String
ren_civ_engineer String
ren_oth_designers String
ren_m_e_engineer String
ren_contractors List of Page See explanation of "lists" above.
ren_cost Float Enter number (dollars) only.
ren_cost_approx Boolean Enter 1 for Yes, leave blank for No.
ren_awards String
demolished Date mm/dd/yyyy OR mm/yyyy OR yyyy.
demolished_approx Boolean Enter 1 for Yes, leave blank for No.
national_register_id Integer
national_register_url URL

The Firms Table

Field Name Type Comments
logo File We recommend leaving this field blank here. See above.
image File We recommend leaving this field blank here. See above.
firm_type List of String See explanation of "lists" above.
founders List of Page See explanation of "lists" above.
antecedents List of Page See explanation of "lists" above.
successors List of Page See explanation of "lists" above.
personnel List of Page See explanation of "lists" above.
location String
founded Start date mm/dd/yyyy OR mm/yyyy OR yyyy.
founded_approx Boolean Enter 1 for Yes, leave blank for No.
dissolved End date mm/dd/yyyy OR mm/yyyy OR yyyy.
dissolved_approx Boolean Enter 1 for Yes, leave blank for No.
awards List of String See explanation of "lists" above.
significant_projects List of Page See explanation of "lists" above.
website URL

The Grounds Table

Field Name Type Comments
alt_name String
image File We recommend leaving this field blank here. See above.
address String
perimeter Text See instructions above for formatting.
Geo Coordinates See instructions above for formatting.
founder List of String See explanation of "lists" above.
owner String
builder List of String See explanation of "lists" above.
landscape_architect Page
other_designers List of Page See explanation of "lists" above.
acreage Float Bare acreage amount here, with no text.
elevation Float
founded Date mm/dd/yyyy OR mm/yyyy OR yyyy.
founded_approx Boolean Enter 1 for Yes, leave blank for No.
opened Date mm/dd/yyyy OR mm/yyyy OR yyyy.
opened_approx Boolean Enter 1 for Yes, leave blank for No.
type List of String See explanation of "lists" above.
NRHP_ID String
NRHP_url URL

The Import

After the CSV file is in a format that allows for one-to-one import to the wiki fields, click on the Import CSV page, select the CSV file using the "Browse" button at the top of the page, leave the encoding type as-is, select "Skip" for pages that already exist, and click on the Import button at the bottom to initiate the process.

Note that wikis do not import files of this type all at one time. To lessen the load on the site, they import one record each time there is another page request anywhere else on the wiki. This serves to spread the backend processing load out and allows the wiki to remain responsive to other users. Over the next few minutes or hours the wiki will import one record at a time from the CSV until all the records are online.