Posts

Storing files on Hive Blockchain: Part 1 - Planning

avatar of @geekgirl
25
gg
@geekgirl
·
·
0 views
·
3 min read

For last couple of weeks I was trying to figure out if it was possible to store files like images, pdf, audio, video, etc on Hive blockchain. Hive is a decentralized database. So, in theory it should be possible. Now the question is, if it would be efficient and useful to do so.

All transactions and data in Hive are stored in blocks. These blocks are limited in size. Each block can have no more than 65,536 bytes. Many large files wouldn't fit into one block. In order to store files larger than block size limit, files would need to be split into pieces while storing. Later when we need to retrieve the files we should be able to put the pieces back together to its original form.

As I was experimenting with various aspects of this project, I ended up writing many small scripts that would perform certain functions. I had to stop at the point where I had to put all the various functions together to achieve the desired outcome. In other words, my plan lacked proper planning. That's why I writing this post to organize my project a little better for future reference and hopefully to get some feedback on how to achieve the goal more efficiently.

Project: Storing Files on Hive blockchain.

Goal of the project:To store image, document, audio, and video files on Hive blockchain. Afterwards to retrieve these files from the blockchain. Then add an ability to share with others.

  • Step 1: Will use Streamlit python module for GUI purposes. Streamlit has built-in file upload functions that easily provide file sizes, types, and names.
  • Step 2: Uploaded files will need to be encrypted. The encrypted version of the file will be split into pieces of about 60,000 bytes. These pieces will be stored on Hive blockchain using custom_json operation. After each storing process, the exact or close to exact location of the pieces will need to be recorded separately. Once all pieces are stored on the blockchain, the list of the locations for each piece will be stored separately with a wallet transfer in any encrypted memo.
  • Step 3: Separate account name will store wallet transfers with encrypted memo that provides more details of where file pieces are stored and in which order.
  • Step 4: Now that we can store files, we need an ability to retrieve the files and read/view them and/or share with others. Separate Streamlit app will be used for this purpose. First to check all the available files stored by a user, we check the dedicated account that keeps track of wallet transfers with encrypted memo in a certain format. Clicking a chosen file will get the locations data from the memo and retrieve the file pieces, put the together and present in its original format.
  • Step 5: First four steps only allow storing and retrieving files by one user. Next step is to make it possible for this user to share the files with other users. One approach can be user1 will send the decryption code to user2, which will give user2 full access to the files. The problem with this is, if user1 changes mind and would like revoke ability by user2 to view the files.
  • Step 6: Only share temporary decryption code with users the files are shared with. After some time the temporary code can expire or can be changed manually by the user sharing the files. I am not sure how it can be achieved yet. One of the ideas is to send partial decryption code to the user files being shared with, and the other half is sent to the App.

I haven't thought through all pieces yet. Not even sure if any of the above steps make sense. I will give a try one step at a time and see what the outcome can be. At the very least, it will be a learning experience for me. If you have any ideas to achieve the goals described above in a more efficient manner, please let me know in the comments.

Posted Using LeoFinance Beta