Ditching a database for flat JSON storage

Last week, I wrote about how I used a static site generator and some PHP to create all of the HTML for the Lean Web Club.

Today, I wanted to talk about how I use flat JSON storage instead of a database to store data.

Let’s dig in!

Directory structure and security

All of my storage files live in a directory that cannot be accessed with an HTTP request. The only way to get to it is via direct server access.

Files on my server can read and write files in the directory, but you can’t get to it directly from a web browser or via an HTTP request.

To do that, I added an .htaccess file to the directory with the following contents…

deny from all

This prevents all traffic from accessing it.

For purposes of this article, I’ll use /keep-out as the directory name. That’s not what it’s actually called on my server.

Reading and writing files with PHP

The file_get_contents() method can be used to read the content of a file as a string. The file_put_contents() method writes data back down.

The json_encode() and json_decode() methods encode and decode strings to and from JSON, respectively. They’re like PHP versions of the JSON.stringify() and JSON.parse() methods.

I created a series of helper functions that I use to get, set, delete, and rename files.

In the get_file() function, I create a $path from the $filename, then check if a file_exists() at that $path.

If so, I decode and return it. Otherwise, I return an empty fallback object.

<?php

/**
 * Get file
 * @param  String  $filename  The filename
 * @param  *       $fallback  Fallback content if the file isn't found
 * @param  Boolean $as_string Return string instead of decoded object
 * @return *                The file content
 */
function get_file ($filename, $fallback = '{}') {

	// File path
	$path = dirname(__FILE__) . '/keep-out/' . $filename . '.json';

	// If file exists, return it
	if (file_exists($path)) {
		$file = file_get_contents($path);
		return json_decode($file, true);
	}

	// Otherwise, return a fallback
	return json_decode($fallback, true);

}

In the set_file() function, I encode a string of $content and create or overwrite a file with the provided $filename.

<?php

/**
 * Create/update a file
 * @param String $filename The filename
 * @param *      $content  The content to save
*/
function set_file ($filename, $content, $fallback = '{}') {

	// File path
	$path = dirname(__FILE__) . '/keep-out/' . $filename . '.json';

	// If there's no content but there's a fallback, use it
	if (empty($content)) {
		file_put_contents($path, $fallback);
		return;
	}

	// Otherwise, save the content
	file_put_contents($path, json_encode($content));

}

To delete a file, I can use the unlink() function, and to rename it, I can use the rename() function.

<?php

/**
 * Delete a file
 * @param String $filename The filename
*/
function remove_file ($filename) {
	if (empty($filename)) return;
	if (!file_exists($filename)) return;
	unlink(dirname(__FILE__) . '/keep-out/' . $filename . '.json');
}

/**
 * Rename a file
 * @param  String $old The old filename
 * @param  String $new The new filename
 */
function rename_file ($old, $new) {
	rename(dirname(__FILE__) . '/keep-out/' . $old . '.json', dirname(__FILE__) . '/keep-out/' . $new . '.json');
}

And this is what I love about PHP, honestly.

It’s got simple building blocks I can use to build whatever I need.

An example: saving user bookmarks

In the Lean Web Club, users can save their favorite learning paths, tutorials, projects, and tools for quick access later.

I’ll talk about how I handle dynamic content on the front end in another article, but here’s how that looks on the back.

First, I use the get_file() function to get the file with their $username from the /keep-out/bookmarks directory. If there’s no file yet, I use an empty array as a fallback.

<?php

$bookmarks = get_file('bookmarks/' . $username, '[]');

Next, I add the new bookmark to the data. Then, I save the file back down to JSON storage.

<?php

$bookmarks = get_file('bookmarks/' . $username, '[]');
$bookmarks[] = $new_bookmark_id;
set_file('bookmarks/' . $username, $bookmarks);

The whole thing is incredibly fast and much more simple than writing SQL database queries. And, if I’m not sure what’s going on, I can just open up a JSON file and look.

Is this for everyone?

Probably not! I’d imagine with incredibly large datasets this has limits.

But for the kind of apps I build, and the number of users I typically manage (low thousands), it’s the perfect solution for my needs.

What about other languages? Does this have to be PHP?

Nope! You can use any server language you prefer. Ruby, Node, and so on all have methods for reading and writing files.

I use PHP because I already know it, and it runs reliably (and comes preinstalled) on pretty much any server I’d use. If you’re more comfortable in Node, by all means use that instead!