The DHS Program User Forum
Discussions regarding The DHS Program data and results
Home » Data » Dataset access » Bulk download only Standard DHS datasets?
Bulk download only Standard DHS datasets? [message #13323] Wed, 18 October 2017 14:56 Go to next message
archju01 is currently offline  archju01
Messages: 2
Registered: October 2017
Location: Washington, DC
Member
The bulk download function is great and has greatly improved our ability to access only the newest datasets for each country of interest. However, my current project is interested in some variables present in the standard DHS but not the MIS. When selecting the most recent survey, the bulk download list does not seem to allow for only the most recent standard DHS. This affects Ghana, Kenya, and others. Is there a way to exclude MIS survey data? Thanks in advance.
Re: Bulk download only Standard DHS datasets? [message #13325 is a reply to message #13323] Thu, 19 October 2017 10:05 Go to previous messageGo to next message
admin is currently offline  admin
Messages: 50
Registered: November 2012
Senior Member
Administrator
Hello, Yes we can add an option to download most recent by survey type: "Most Recent DHS"; "Most Recent SPA"; "Most Recent AIS", and "Most Recent MIS". We will include this in our next set of updates to the bulk download tool which will likely happen within the new few weeks.
Thank you for the suggestion!

DHS UserForum Administrator


We now have repositories of code written in Stata and SPSS available on Github. Please reference these code repositories as a resource for code for matching or calculating DHS indicators. The code repositories can be found at:

https://github.com/DHSProgram/DHS-Indicators-Stata
https://github.com/DHSProgram/DHS-Indicators-SPSS

[Updated on: Tue, 31 October 2017 13:07]

Report message to a moderator

Re: Bulk download only Standard DHS datasets? [message #13327 is a reply to message #13325] Thu, 19 October 2017 11:30 Go to previous messageGo to next message
archju01 is currently offline  archju01
Messages: 2
Registered: October 2017
Location: Washington, DC
Member
Thank you for being so quick to reply and accommodate. I look forward to using the new function when it's online.
Re: Bulk download only Standard DHS datasets? [message #14346 is a reply to message #13323] Mon, 26 March 2018 12:45 Go to previous messageGo to next message
cportner is currently offline  cportner
Messages: 20
Registered: September 2014
Location: Seattle
Member
When is this feature going to roll out? It would be great if there was a way to download only DHS datasets (not just the most recent one).
Re: Bulk download only Standard DHS datasets? [message #14347 is a reply to message #14346] Mon, 26 March 2018 13:00 Go to previous messageGo to next message
admin is currently offline  admin
Messages: 50
Registered: November 2012
Senior Member
Administrator
Yes, this feature is still on the list of items for next release. Should be available by middle of next month.

Sorry for the delay.

DHS UserForum Administrator


We now have repositories of code written in Stata and SPSS available on Github. Please reference these code repositories as a resource for code for matching or calculating DHS indicators. The code repositories can be found at:

https://github.com/DHSProgram/DHS-Indicators-Stata
https://github.com/DHSProgram/DHS-Indicators-SPSS

Re: Bulk download only Standard DHS datasets? [message #14349 is a reply to message #14347] Mon, 26 March 2018 17:37 Go to previous messageGo to next message
cportner is currently offline  cportner
Messages: 20
Registered: September 2014
Location: Seattle
Member
Wonderful! Thank you for the quick reply.

In the meantime, is there a way to separate out which files are standard DHS and which files are other types? I am using a lot of countries for my current project, and I am trying to automate the read-in process as much as possible.

From what I can tell the files follow an identical naming pattern across standard DHS and MIS. Also, a quick look at the survey data reveals no indicator for which type of survey it was. Hence, it appears that the only option currently is to read in the data set, check whether all the variables I need are available, and drop the data set if they are not. Is there another way to do this?

Claus
Re: Bulk download only Standard DHS datasets? [message #14357 is a reply to message #14349] Tue, 27 March 2018 07:45 Go to previous messageGo to next message
Bridgette-DHS is currently offline  Bridgette-DHS
Messages: 3016
Registered: February 2013
Senior Member
You can go to the survey search and use the "survey type" criteria to get a list of the surveys you are interested in. You have a complete breakdown of survey types here.

[Updated on: Tue, 27 March 2018 07:47]

Report message to a moderator

Re: Bulk download only Standard DHS datasets? [message #17639 is a reply to message #14357] Wed, 01 May 2019 15:03 Go to previous messageGo to next message
cportner is currently offline  cportner
Messages: 20
Registered: September 2014
Location: Seattle
Member
I tried the survey search function, but that does not appear to give you an option to download all the files directly. Would it be possible to expand the "Download Datasets Using a Download Manager" option to include only standard DHS surveys?

All the best,

Claus
Re: Bulk download only Standard DHS datasets? [message #17641 is a reply to message #17639] Wed, 01 May 2019 15:13 Go to previous messageGo to next message
admin is currently offline  admin
Messages: 50
Registered: November 2012
Senior Member
Administrator
Hello Claus, I believe what you are asking for is the "All DHS" option in the download manager.

The DHS Program User Forum Administrator


We now have repositories of code written in Stata and SPSS available on Github. Please reference these code repositories as a resource for code for matching or calculating DHS indicators. The code repositories can be found at:

https://github.com/DHSProgram/DHS-Indicators-Stata
https://github.com/DHSProgram/DHS-Indicators-SPSS

Re: Bulk download only Standard DHS datasets? [message #17645 is a reply to message #17641] Wed, 01 May 2019 17:40 Go to previous messageGo to next message
cportner is currently offline  cportner
Messages: 20
Registered: September 2014
Location: Seattle
Member
The problem with that one is that it also includes the once classified as "special DHS". For example, the two from the Dominican Republic with samples living in sugar cane planations are included if I use this approach. What I am looking for it an option to select *only* standard DHS surveys.
Re: Bulk download only Standard DHS datasets? [message #17646 is a reply to message #17645] Wed, 01 May 2019 17:58 Go to previous messageGo to next message
admin is currently offline  admin
Messages: 50
Registered: November 2012
Senior Member
Administrator
I see. We will look into adding the option. In the meantime can you manually remove those datasets you don't want once downloaded?

The DHS Program User Forum Administrator



We now have repositories of code written in Stata and SPSS available on Github. Please reference these code repositories as a resource for code for matching or calculating DHS indicators. The code repositories can be found at:

https://github.com/DHSProgram/DHS-Indicators-Stata
https://github.com/DHSProgram/DHS-Indicators-SPSS

Re: Bulk download only Standard DHS datasets? [message #17648 is a reply to message #17646] Wed, 01 May 2019 21:10 Go to previous messageGo to next message
cportner is currently offline  cportner
Messages: 20
Registered: September 2014
Location: Seattle
Member
That is essentially what I have been doing. The problem is that the download consists of 286 separated zip files, so I have tried to automate the process. What I have done is automatically drop any surveys that are missing a specific set of variables such as v602 and bidx_01 (or rather have all missing observations for these variables). This, however, does not get rid of some special DHS, such as the Dominican Republic ones on sugar plantation populations.
Re: Bulk download only Standard DHS datasets? [message #17654 is a reply to message #17648] Thu, 02 May 2019 08:34 Go to previous messageGo to next message
admin is currently offline  admin
Messages: 50
Registered: November 2012
Senior Member
Administrator
I see the issue you are having. You could also parse the file name to remove or skip the files you dont want. Each file is coded with the country code and version number that uniquely identifies a survey.

That might be messy coding but it should work.

However I do agree adding an option to just select standard surveys is good idea and probably not too complicated. We will look into this.

The DHS Program User Forum Administrator



We now have repositories of code written in Stata and SPSS available on Github. Please reference these code repositories as a resource for code for matching or calculating DHS indicators. The code repositories can be found at:

https://github.com/DHSProgram/DHS-Indicators-Stata
https://github.com/DHSProgram/DHS-Indicators-SPSS

[Updated on: Thu, 02 May 2019 08:35]

Report message to a moderator

Re: Bulk download only Standard DHS datasets? [message #17656 is a reply to message #17654] Thu, 02 May 2019 12:02 Go to previous messageGo to next message
cportner is currently offline  cportner
Messages: 20
Registered: September 2014
Location: Seattle
Member
Thank you! It would be great if the option to just select standard DHS was available.

Until then, what I am currently doing is close to what you describe. I take the country code and version number from both the file name and from within the data set itself (since that helps catch/remove the individual Indian state files). The script uses those to select the relevant variables by phase.

Is there a way to identify whether a given survey comes from a special DHS or a standard DHS from within the data? It appears that the version numbering is in chronological order, so, for example, the Dominican Republic special DHS are named 5A and 6A because they came after the standard ones for the respective phases. Alternatively, is there a file that identifies surveys as either special or standard DHSs?

[Updated on: Thu, 02 May 2019 12:12]

Report message to a moderator

Re: Bulk download only Standard DHS datasets? [message #17678 is a reply to message #17656] Mon, 06 May 2019 11:05 Go to previous message
admin is currently offline  admin
Messages: 50
Registered: November 2012
Senior Member
Administrator
Hi Claus, I think what you are doing is the best workaround for now. We do plan to add an option for selecting standard only surveys. Not sure exactly when that will get done but not too distant future.

Thank you for the suggestion.

The DHS Program User Forum Administrator


We now have repositories of code written in Stata and SPSS available on Github. Please reference these code repositories as a resource for code for matching or calculating DHS indicators. The code repositories can be found at:

https://github.com/DHSProgram/DHS-Indicators-Stata
https://github.com/DHSProgram/DHS-Indicators-SPSS

Previous Topic: Height and Weight for men and women
Next Topic: Bulk Downloading of Datasets Using a Download Manager
Goto Forum:
  


Current Time: Thu Mar 28 09:41:33 Coordinated Universal Time 2024