Transfer to IPFS/Filecoin • MuckRock

Transfer to IPFS/Filecoin • MuckRock

At DocumentCloud, with the support of the Filecoin Foundation for the Decentralized Web, we upload documents for long-term storage and distribution via the Filecoin network and the InterPlanetary File System (IPFS).

IPFS is a peer-to-peer distributed file system, meaning files are distributed across many hosts, also called nodes. The Filecoin network is a system that incentivizes sharing and storing data using a blockchain and tokens, also through IPFS. Both were developed by Protocol Labs, an open-source software development company.

One of the main strengths of the IPFS network is its censorship resistance: because it is decentralized, authorities cannot target individual entities or locations to take a file offline. It also provides backup, so your documents are much less likely to go offline, as in the case of a DDoS attack.

IPFS can also be a powerful privacy resource, as it can be difficult to trace documents back to their source. However, DocumentCloud is not an anonymous service, so if you are interested in anonymity, you can check out other resources, such as this guide from Reporters Without Borders.

Another benefit is the ability to check if content has been modified. IPFS uses unique Content Identifiers (CIDs) to check if a file has been tampered with. No matter where or how you get a document, if you know its CID, you can check if it has been modified.

The decentralized upload structure has its drawbacks: it’s much harder to remove a document when you no longer want it to be public. If you upload a file to an IPFS node, any other node can copy and store it – so it remains available until every node on the network unbinds (or removes) it.

MuckRock is committed to working with its users to keep every DocumentCloud document live, so we feel it’s important to leverage the IPFS network as much as possible and make sure our users understand the trade-offs.

Since March 2022, we have uploaded nearly 800,000 documents using DocumentCloud’s Push to IPFS/Filecoin add-on. If you use this add-on for your own documents, we will use a service called web3.storage to push them to the IPFS/Filecoin network. We will also add tags to your documents that list the CID and IPFS URL. The IPFS URL will allow you to easily access the document from your web browser through an IPFS gateway.

We regularly receive cease and desist letters and have sometimes had to make difficult decisions about whether to keep documents available online or our tools. Some of these cease and desist letters are clearly false, but others are legally sound and we work with our users and the legal system to keep these documents available to the public.

Last winter, MuckRock received a takedown request for documents about the Association of Appin Training Centers and its “hacking for hire” services. The request referenced a preliminary court ruling from New Delhi against Reuters for its original story about Appin, which has since been replaced with an editor’s note. With help from the Electronic Frontier Foundation, we were able to resist this takedown request and continue to host these documents on DocumentCloud.

Last year, we were informed by Netlify, the service that hosts some of our frontend code, that a document on DocumentCloud violated a French anti-terrorism law. The document, an issue of “Inspire” published by al-Qaeda in the Arabian Peninsula, had appeared in Jeremy Scahill’s reporting on the Yemeni al-Qaeda affiliate.

In this case, we coordinated with The Intercept’s editorial staff, which chose to keep the document private rather than challenge it in the French court. But our alternative would have been to convince a court to keep the document online – or risk DocumentCloud being shut down entirely.

Many of the deletion requests we receive concern documents of journalistic importance that we want to keep available long-term, regardless of the whims of government or business actors. We believe the IFPS network can help us achieve this goal.

Whether you want to experiment with an IPFS node, back up your documents for long-term retention, or migrate them to a censorship-resistant repository, the Push to IPFS add-on is a service you might find useful. The add-on is also open source, so if you want to fork or reuse the code for your own purposes, you can find it in this MuckRock Github repository.

Leave a Reply

Your email address will not be published. Required fields are marked *