Development Experience Clearinghouse (USAID)

The Development Experience Clearinghouse (DEC) serves as a repository for hundreds of thousands of documents for USAID projects. USAID projects submit evaluations and other related documents to the DEC to share knowledge throughout the agency and its partners. 

 

Case Study

Problem:

Projects submit documents such as evaluations for their projects and that kicks off a whole process where the content team has to categorize these documents. The project desperately needed automation to keep up with the volume of documents being submitted on a daily basis. The content team was really small and 

Solution:

This project had a ton of moving parts. 

Firstly, the public site: dec.usaid.gov was a built on a .Net based CMS called Presto, the off the shelf solution for Text Analytics was built in Java. The first goal was to develop middleware that connected the two applications. 

The basic workflow was when a user submits a new document to the site, it kicks off a process where that document is ran through a text analytics application where it contains a variety of modules based on various categorization profiles. Those profiles included Geographic Descriptors, Authors, Language Detector, Primary subjects and other taxonomies. After documents are ran through those profiles, the information was compiled into an XML and uploaded to the Presto API. 

To minimize server load we batched files that were uploaded and ran these profiles overnight and emailed us results that next morning to review if there were any errors. This project was a huge value for the content team to stay up to date with the constant submissions. 

Other tasks included implementing a google search appliance which allowed for a more robust search experience, redesign of homepage, connection with the Kaltura API to serve media assets on the site, an automated solution to hide restricted PDF's and developed the DEC API for public entities to access the DEC's data.