Data Sharing Systems - Why Is Data Sharing Hard? | Part 3 of 7

Even a well-intentioned data sharing system faces a long list of technical and operational hurdles. Here are some of the most significant:

• Privacy. How will a data sharing system manage nonprofit privacy? Although General Data Protection Regulations (GDPR) is a European standard, it's worth designing with it in mind - California's privacy rules and likely future U.S. standards will probably follow its lead. Will the system be a data processor or a data controller under these frameworks? How will conflicting rights be managed if a nonprofit grants different rights to different grantmakers? How will data aging rules be established when different grantmakers have different data retention requirements - especially for sensitive information like FAFSA data on scholarship applications?
• Security. What mechanisms will protect the system from unauthorized access? What user roles and authorities will exist? How will identity be verified when a nonprofit needs to transfer control of its information to a new contact - something we see happen frequently as a Grant Management System (GMS) provider? Will multiple nonprofit users be allowed to work on the same information, or will each maintain their own version? If the latter, what rules will determine which version gets shared?
• Availability. Will the system need to go down for updates? How will downtime be coordinated with grantmakers who may be in the middle of an application cycle? Who monitors the system and responds to failures? What happens if the system goes offline during a critical deadline window - do GMS providers collect information locally and sync later?
• Maintenance. How will code updates be communicated and tested across GMS providers? Changes to schemas, data types, allowable values, and API protocols all have downstream effects. Will backward compatibility be maintained so providers who haven't updated yet don't experience breaking changes?
• Schema authority. Who owns the data schema? How are conflicts resolved? How will the system handle the inevitable divergence in how different grantmakers ask similar questions - one using an enumerated list, another using free text, each with different character limits? Changing a field's data type or allowable values after the system is in use is very difficult. Text changes may be manageable; structural changes rarely are.
• Data authority. If the same data field is updated by three different sources - the nonprofit directly, one GMS provider, and another GMS provider - which version is authoritative? Should any of these updates be "sticky" and override future changes? How does this interact with a nonprofit's right to correct their own information?
• Data conflict resolution. What happens when conflicting data arrives for the same field from two different sources? This is especially likely if flat files are the initial method of data exchange, since the lag between updates increases the chance of collision. Strategies like "last write wins" or data merging each have tradeoffs, and whatever approach is chosen, there needs to be a way to notify the GMS provider and the nonprofit - and potentially let the nonprofit resolve the conflict themselves.
• User management. Who supports nonprofits that want to exercise their privacy rights or update their organizational contacts? Who manages the administrative users of the platform itself?
• Data curation. Nonprofits frequently answer the same question differently for different grantmakers - tailoring their response to what a particular funder cares about. If those answers are stored in a shared system, which version gets shared? Is there a way to maintain multiple versions of the same answer for different contexts?
• Data fields. How do you uniquely and unambiguously identify a field that a GMS provider wants to retrieve? Simple labels aren't enough - there can be typos, subtle wording differences, or the same question formatted differently across grantmakers. Where is validation performed? How are organizations without tax ID numbers (those operating under a fiscal agent) handled? How do contact fields accommodate the many different types of contacts in a single application - organization head, project lead, fiscal agent, recommenders? If GMS providers are expected to absorb many of these complexities themselves, that means significant development cost, which will directly affect their willingness to participate.

Taken together, these aren't just technical challenges - they are governance, policy, and organizational challenges. The technology is often the easier part.