Improving Descriptive Practices for Born-Digital Material in an Archival Context

by Annie Tummino

In 2014/15 I worked at the New York Metropolitan Library Council (METRO) as the Project Manager for the National Digital Stewardship Residency (NDSR) program in New York City, providing administrative support for five fantastic digital stewardship projects. While I gained a great deal of theoretical knowledge during that time, my hands-on work with born digital materials has been fairly minimal. When I saw that METRO was offering a workshop on “Improving Descriptive Practices for Born-Digital Material in An Archival Context” with Shira Peltzman, former NDSR-NY resident (and currently Digital Archivist for UCLA Library Special Collections), I jumped at the opportunity to sign-up.

For the last two years I have served as the archivist at SUNY Maritime College, working as a “lone arranger” on a small library staff. My emphasis has been on modernizing the technical infrastructure for our special collections and archives. We’ve implemented ArchivesSpace for collections management and are in the process of launching a digital collections platform. However, most of my work with born-digital materials has been of a very controlled type; for example, oral histories and student photographs that we’ve collected as part of documentation projects. While we don’t routinely accession born-digital materials, I know the reckoning will occur eventually. A workshop on descriptive practices seemed like a good place to start.

Shira emphasized that a great deal of technical and administrative metadata is captured during processing of born-digital materials, but not all of this information should be recorded in finding aids. Part of the archivist’s job is figuring out which data is meaningful for researchers and important for access. Quoting Bertram Lyons, she also emphasized that possessing a basic understanding of the underlying “chemistry” of digital files will help archivists become better stewards of born-digital materials. To that end, she started the day with a “digital deep dive” outlining some of this underlying digital chemistry, including bits and bytes, character encoding, and plain text versus binary data. This was followed by an activity where we worked in small groups to analyze the interdependencies involved in finding, retrieving, and rendering the contents of files in given scenarios. The activity definitely succeeded in demonstrating the complexities involved in processing digital media, and provided an important foundation for our subsequent discussion of descriptive practice.

The following bullets, pulled directly from Shira’s presentation, succinctly summarize the unique issues archivists face when processing born digital materials:

  • Processing digital material often requires us to (literally) transform the files we’re working with
  • As stewards of this material, we must be prepared to record, account for, and explain these changes to researchers
  • Having guidelines that helps us do this consistently and uniformly is essential

The need for transparency and standardization were themes that came up again and again throughout the day.

To deal with some of the special challenges inherent in describing born-digital materials, a working group under the aegis of the UC Born-Digital Content Common Knowledge Group (CKG) has developed a UC-wide descriptive standard for born-digital archival material. The elements map to existing descriptive standards (DACS, EAD, and MARC) while offering additional guidance for born-digital materials where gaps exist. The most up-to-date version is on GitHub, where users can make pull requests to specific descriptive elements of the guidelines if they’d like to propose revisions. They have also been deposited in eScholarship, the institutional repository for the UC community.

Working in small groups, workshop participants took a closer look at the UC guidelines, examining particular elements, such as Processing Information; Scope and Content; Physical Description; and Extent. Drawing from our experience, we investigated differences and similarities in using these elements for born-digital materials in comparison to traditional materials. We also discussed the potential impacts of skipping these elements on the research process. We agreed that lack of standardization and transparency sows confusion, as researchers often don’t understand how born-digital media can be accessed, how much of it there is, or how it relates to the collection as a whole.

For our final activity, each group reviewed a published finding aid and identified at least five ways that the description of born-digital materials could be improved in the finding aid. The collections represented were all hybrid, containing born-digital materials as well as papers and other analog formats. It was common for the digital materials to be under-described, with unclear access statuses and procedures. The UC guidelines were extremely helpful in terms of generating ideas for improvements. However, the exercise also led to real talk about resource limitations and implementation. How do born-digital materials fit into an MPLP context? What do the guidelines mean for description in terms of tiered or efficient processing? No solid answers here, but great food for thought.

On the whole, the workshop was a great mix of presentation, discussion, and activities. I left with some immediate ideas to apply in my own institution. I hope more archivists will have opportunities to take workshops like this one and will check out the UC Guidelines.


Tummino Picture

Annie Tummino is the Archivist & Scholarly Communications Librarian at SUNY Maritime College, where she immerses herself in maritime special collections and advocates for Open Access while working in a historic fort by the sea. She received her Masters in Library and Information Studies and Archives Certificate from Queens College-CUNY in December, 2010.

Embedded Archives at the Institute for Social Research

by Kelly Chatain

This is the fourth post in the BloggERS Embedded Series.

As any archivist will tell you, the closer you can work with creators of digital content, the better. I work for the Institute for Social Research (ISR) at the University of Michigan. To be more precise, I am a part of the Survey Research Center (SRC), one of five centers that comprise the Institute and the largest academic social science research center in the United States. But really, I was hired by the Survey Research Operations (SRO) group, the operational arm of SRC, that conducts surveys all over the world collecting vast and varied amounts of data. In short, I am very close to the content creators. They move fast, they produce an extraordinary amount of content, and they needed help.

Being an ‘embedded’ archivist in this context is not just about the end of the line; it’s about understanding and supporting the entire lifecycle. It’s archives, records management, knowledge management, and more, all rolled into one big job description. I’m a functional group of one interacting with every other functional group within SRO to help manage research records in an increasingly fragmented and prolific digital world. I help to build good practices, relationships, and infrastructure among ourselves and other institutions working towards common scientific goals.

Lofty. Let’s break it down a bit.

Find it, back it up, secure it

When I arrived in 2012, SRO had a physical archive of master study files that had been tended to by survey research staff over the years. These records provide important reference points for sampling and contacting respondents, designing questionnaires, training interviewers, monitoring data collection activities, coding data, and more. After the advent of the digital age, a few building moves, and some server upgrades, they also had an extensive shared drive network and an untold number of removable media containing the history of more recent SRO work. My first task was to centralize the older network files, locate and back up the removable media, and make sure sensitive data was out of reach. Treesize Professional is a great tool for this type of work because it creates detailed reports and clear visualizations of disk space usage. This process also produced SRO’s first retention schedule and an updated collection policy for the archive.

Charts produced by Treesize Professional used for the initial records survey and collection.
A small selection of removable media containing earlier digital content.

Welcome, GSuite

Despite its academic home, SRO operates more like a business. It serves University of Michigan researchers as well as external researchers (national and international), meeting the unique requirements for increasingly complex studies. It maintains a national field staff of interviewers as well as a centralized telephone call center. The University of Michigan moved to Google Apps for Education (now GSuite) shortly after I arrived, which brought new challenges, particularly in security and organization. GSuite is not the only documentation environment in which SRO operates, but training in the Googleverse coincided nicely with establishing guidance on best practices for email, file management, and organization in general. For instance, we try to label important emails by project (increasingly decisions are documented only in email) which can then be archived with the other documentation at the end of the study (IMAP to Thunderbird and export to pdf; or Google export to .mbox, then into Thunderbird). Google Drive files are downloaded to our main projects file server in .zip format at the end of the study.

Metadata, metadata, metadata

A marvelous video on YouTube perfectly captures the struggle of data sharing and reuse when documentation isn’t available. The survey data that SRO collects is delivered to the principal investigator, but SRO also collects and requires documentation for data about the survey process to use for our own analysis and design purposes. Think study-level descriptions, methodologies, statistics, and more. I’m still working on finding that delicate balance of collecting enough metadata to facilitate discovery and understanding while not putting undue burden on study staff. The answer (in progress) is a SQL database that will extract targeted structured data from as many of our administrative and survey systems as possible, which can then be augmented with manually entered descriptive metadata as needed. In addition, I’m looking to the Data Documentation Initiative, a robust metadata standard for documenting a wide variety of data types and formats, to promote sharing and reuse in the future.

DDI is an international standard for describing data.

Preserve it

The original plan for digital preservation was to implement and maintain our own repository using an existing open-source or proprietary system. Then I found my new family in the International Association for Social Science Information Services & Technology (IASSIST) and realized I don’t have to do this alone. In fact, just across the hall from SRO is the Inter-University Consortium for Political and Social Research (ICPSR), who recently launched a new platform called Archonnex for their data archive(s). Out of the box, Archonnex already delivers much of the basic functionality SRO is looking for, including support for the ever-evolving  preservation needs of digital content, but it can also be customized to serve the particular needs of a university, journal, research center, or individual department like SRO.

Searching for data in OpenICPSR, built on the new Archonnex platform.


The embedded archivist incorporates a big picture perspective with the specific daily challenges of managing records in ways that not many positions allow. And you never know what you might be working on next…

Kelly Chatain is Associate Archivist at the Institute for Social Research, University of Michigan in Ann Arbor. She holds an MS from Pratt Institute in Brooklyn, NY.

Thoughts from a Newcomer: Code4Lib 2018 Recap

by Erica Titkemeyer

After prioritizing audiovisual preservation conferences for so long, this year I chose to attend my first Code4Lib annual conference in Washington, D.C. I looked forward to the talks most directly related to my work, but also knew that there would be many relatable discussions to participate in, as we are all users/creators of library technology tools and we all seek solutions to similar data management challenges.

The conference started with Chris Bourg’s keynote, detailing research on why marginalized individuals are compelled to leave their positions in tech jobs because of undue discrimination. Calling for increased diversity in the workplace, less mansplaining/whitesplaining, more vouching for and amplification of marginalized colleagues, Dr. Bourg set out to “equip the choir”. She also pointed out that Junot Diaz said it best at ALA’s midwinter conference, when he called for a reckoning, explaining that “A profession that is 88% white means 5000% agony for people of color, no matter how liberal and enlightened you think you are.”

I appreciated her decision to use this opportunity to point out our own shortcomings, contradictions and need to do better. I will also say, if you ever needed proof that there is an equity problem in the tech world, you can:

  1. Listen to Dr. Bourg’s talk and
  2. Read the online trolling and harassment that she has since been subjected to because of it.

Since the backlash, Code4Lib has released a Community Statement in support of her remarks.

Following the keynote, the first round of talks further assured me that I had chosen the right conference to attend. In Andreas Orphanides’ talk: “Systems thinking: a practical field guide”, he cleverly pointed out system failures and hacks we all experience in our daily lives, and how they are analogous to the software we might build and where there might be areas for improvement. I also appreciated Julie Swierczek’s talk “For Beginners – No Experience Necessary”, in which she made the case for improving how we teach to true beginners in workshops. She also argued that instructors should not assume everyone is on the same level-playing field just because the title includes “for beginners” as it is not likely that attendees will know how to self-select workshops, especially if they are truly beginners to the technology being taught.

As a fan of Arduino (an open source hardware and software platform that supports DIY electronic prototying), I was curious to hear Monica Maceli’s “Low-cost preservation Environment Monitoring” talk, where she described her experience developing an environmental datalogger using the Raspberry Pi (similar microcontroller concept to Arduino) comparing the results and associated costs with a commercial datalogger, the PEM2. While it would require staff with appropriate expertise, it seemed to be a worthwhile endeavor for anyone wishing to spend a quarter of the price.

With the sunsetting of Flash, I was eager to hear how Jacob Zaborowski’s talk “Save Homestar Runner!: Preserving Flash on the Web” would address the preservation concerns surrounding Homestar Runner, an online cartoon series that began in 2000 using flash animation. Knowing that tools such as Webrecorder and Archive-it would capture, but not aid in preserving the SWF files comprising the animations, Zaborowski sought out free and/or open source tools for transcoding the files to a more accessible and preservation-minded format. Like many audiovisual-based digital formats, tools for transcoding the SWF files were not entirely reliable or capable of migrating all of the unique attributes to a new container with different encodings. At the time of his talk, the folks at Homestar Runner were in the midst of a site redesign to hopefully resolve some of these issues.

While I don’t have the space to summarize all of the talks I found relatable or enlightening during my time at Code4Lib, I think these few that I’ve mentioned show how varied the topics can be, while still managing to complement the information management work we are all charged with doing.

TitkemeyerErica.jpgErica Titkemeyer is the Audiovisual Conservator for the Southern Folklife Collection (SFC) at the University of North Carolina at Chapel Hill. She is the Project Director on the Andrew W. Mellon Foundation grant Extending the Reach of Southern Audiovisual Sources, and overseas the digitization, preservation and access of audiovisual recordings for the SFC.

Embedded Memory

by Michelle Ganz

This is the third post in the BloggERS Embedded Series.

As an archivist in a small corporate repository for an architecture, design, and consulting firm I have a unique set of challenges and advantages. By being embedded with the creators as they are creating, I have the opportunity to ensure that archival standards are applied at the point of creation rather than after the collection has been transferred to a repository.

My environment is especially unique in the types of digital files I’m collecting. From the architecture side, William McDonough + Partners, I acquire architectural project files (CAD files), sketches, renderings, photographs and video of project phases, presentations, press, media files surrounding the project, and other associated materials. Many of these files are created on specialized, and expensive, software.

From McDonough Innovation, William McDonough’s sustainable development advising firm, I collect the world-changing ideas that Mr. McDonough is developing as they evolve. Mr. McDonough is a global thought leader who provides targeted ideas, product concepts, and solutions to a wide range of sustainable growth issues faced by corporate officers, senior executives, product designers, and project managers. He often works with CEOs to set the vision, and then with the management team to set goals and execute projects. His values-driven approach helps companies to embed sustainable growth principles into their corporate culture and to advance progress toward their positive vision. Archiving an idea is a multi-faceted endeavor. Materials can take the form of audio notes, sketches in a variety of programs, call or meeting recordings, and physical whiteboards. Since my role is embedded within the heart of Mr. McDonough’s enterprises, I ensure that all the right events are captured the right way, as they happen. I gather all the important contextual information and metadata about the event and the file. I can obtain permissions at the point of creation and coordinate directly with the people doing the original capture to ensure I get archival quality files.

Challenges in this environment are very different than what my academic counterparts face. In the academic world there was a chain of leadership that I could advocate to as needed. In my small corporate world there is no one to appeal to once my boss makes up their mind. Corporate interests are all focused on ROI (return on investment), and an archival department is a financial black hole; money is invested, but they will never see a financial return. This means that every new project must show ROI in more creative ways. I focus on how a project will free up other people to do more specialized tasks. But even this is often not enough, and I find myself advocating for things like file standards and server space. Many of the archival records are videos of speeches, events, meetings, or other activities and take up a huge amount of server space. A single month’s worth of archival files can be as large as 169 GB. In the academic setting where the archives is often a part of the library, the IT department is more prepared for the huge amounts of data that come with modern libraries; folding the archival storage needs into this existing digital preservation framework is often just a matter of resource allocation or funds.

Also, nearly every archival function that interacts with outside entities requires permissions these firms are not used to giving. Meetings can include people from 3 or 4 companies in 4 or 5 countries with a variety of NDAs in place with some, but not all, of the parties. In order to record a meeting I must obtain permission from every participant; this can be rather complicated and can create a lot of legal and privacy issues. A procedure was put in place to request permission to record when meetings are set up, as well as when meetings are confirmed. A spreadsheet was created to track all of the responses. For regular meeting participants annual permissions are obtained. This procedure, while effective, is time-consuming. For many meeting participants they are unfamiliar with what an archive is. There are many questions about how the information will be used, stored, disseminated, and accessed.  There are also a lot of questions around the final destination of the archive and what that means for their permissions. To help answer these questions I created fact sheets that explain what the archives are, how archival records are collected and used, deposit timelines, copyright basics, and links to more information. To further reassure participants, we give them the option of asking for a meeting to be deleted after the fact.

This is the server stack for the archives and the two firms. The archive uses 2 blades.
This hub connects the archive to workstations and handles the transfer of TBs of data.

Preservation and access are unique challenges, especially with the architecture files. Many of the project-related files are non-traditional file formats like .dwg, .skb, .indd, .bak, et al., and are created in programs like AutoCAD and SketchUp Pro. I work with the IT department to ensure that the proper backups are completed. We back up to a local server as well as one in the city, but offsite, and a third dark archive in California. I also perform regular checks to confirm the files can open. Due to the fact that projects are often reopened years later, it is impractical to convert the files to a more standardized format. To ensure some level of access without specialized terminals, final elements of the project are saved in a .pdf format. This includes final drawings/renderings and presentations.

Furthermore, I often find myself in the awkward position of arguing with records creators in favor of keeping files that they don’t want but I know have archival value. Without the benefit of patrons, and their potential needs, I am arguing for the future needs of the very people I am arguing with! Without a higher level of administration to appeal to, I am often left with no recourse but to do things that are not in the best interests of the collection. This leads to the unfortunate loss of materials but may not be as bad as it first appears. When considering how traditional archival collections are created and deposited, it is well within reason that these items would never have made it into the collection. I like to think that by being embedded in the creation process, I am able to save far more than would otherwise be deposited if the creators were left to make appraisal decisions on their own.

Michelle Ganz is the Archives Director at McDonough Innovation and the former archivist at Lincoln Memorial University. She received her MILS from the University of Arizona and a BA from the Ohio State University. In addition to her passion for all things archival Michelle loves to cook, read, and watch movies.