↩︎ back to homepage

Setting Up Reading Progress, Notes, and Highlights Sync

A month ago, on a Saturday night when I couldn’t fall asleep, I sat down at my computer to finish configuring my reading infrastructure. The one thing missing for complete satisfaction was syncing my reading progress, notes, and highlights.

I started with reading progress synchronization. KOReader has a dedicated app for this, ready to deploy easily using Docker. However, since I use shared hosting, that option was out of the question. This ruled out not only the official app but also a second one I had found. Fortunately, it turned out that someone had recently run into a similar problem and implemented the sync API in a simple PHP project with no external dependencies. Setting it up and configuring it took about fifteen minutes. The whole process involved creating and securing a domain (Let’s Encrypt), uploading the app to the server, and registering a new user (directly from the browser). After that, I blocked the registration option and made sure the directory containing the SQLite database was not accessible from the browser. It turned out it was accessible. To prevent this, all it took was creating an .htaccess file in the database directory with the following content:

<IfVersion < 2.4>
    Deny from all
</IfVersion>
<IfVersion >= 2.4>
    Require all denied
</IfVersion>

The two conditional blocks above do the same thing, just for different versions of Apache: they deny access to files by returning a 403 (Forbidden) response code. Once I confirmed the app was secured, it was time to configure the sync on my Kindle. To do this, I had to open any book (configuration cannot be done from KOReader’s home screen), then tap the top part of the screen, tap the wrench-and-screwdriver icon, and select “Progress sync”. Then click “Custom sync server” and enter your domain name. After that, go to “Login/register” and enter the username and password of the user you created earlier on your newly set up site. Also check “Automatically sync documents” so you don’t have to do it manually. One important note — if you already have books in progress on the device, their reading progress will not be synced until you start reading them again, or do it manually by opening each one and clicking Settings > Progress sync > Save current document progress from this device. That’s it, the sync should now be working. At the moment, the only downside of phpkoreadersync is the language of the web interface (Dutch). However, this is not a significant drawback for two reasons: first, after the initial setup you’ll barely ever use the web interface again; second, ignoring automatic page translation options (either built into the browser or via plugins), the interface is so straightforward that you don’t need to know the language to operate it.

Syncing notes and highlights is a separate topic. They are stored in sidecar files (.sdr) kept alongside the books they relate to. One way to share them between devices is simple file synchronization, which KOReader supports natively. You set it up by clicking Settings > Cloud storage. You should see a page showing your configured sync servers. In my case it was empty. To configure a new drive, click the plus icon in the top left corner of the page. The available options are: Dropbox, FTP, WebDAV. I went with the last one, since I host my own Nextcloud instance which supports this form of sync, so I didn’t need to set up any additional service. In the home directory of my Nextcloud instance, I created a new folder (“KOReader”) to use as the sync location for books and their associated sidecar files. Since I have two-factor authentication configured in Nextcloud, I had to generate a new app password in the Nextcloud interface by going to Settings > Security and then the “Devices & sessions” section. There, enter an app name (e.g. “KOReader WebDAV”) and click “Generate new app password”, after which a new window will show a username and password — save these somewhere (ideally in a password manager) as you’ll need them shortly (and in the future). With the new app password in hand, go back to KOReader and fill in the WebDAV connection form fields:

  • server name (e.g. “Nextcloud”),
  • WebDAV address (found in your Nextcloud instance — go to the “Files” tab, click Files settings in the bottom left corner, and navigate to the “WebDAV” section where you’ll find the “WebDAV URL” — if you created a dedicated directory to use for sync, append its name at the end, resulting in something like “https://myinstance.com/remote.php/dav/files/krystian/KOReader"),
  • username and password (saved earlier), and
  • starting folder (path to the directory you have designated as the home directory in KOReader where your books are stored, in my case “/documents”)

then click “Save”. A new entry should appear in the list. Unfortunately, in my case clicking it gave me the message “Failed to retrieve folder list. Check configuration and network connection”. I first checked whether I could connect to my directory using those credentials from my laptop. The credentials turned out to be correct. Only after a moment did I understand my mistake — the “starting folder” did not refer to a location on my Kindle, but rather pointed to the WebDAV starting directory. After changing that parameter to “/” the connection worked without issue and I was able to see all the files from my Nextcloud instance in that directory. Unfortunately, it turned out I had made another mistake — quite a significant one. The “Cloud storage” option does nothing more or less than connect to a network drive and save files from that drive onto the Kindle. There is no option to sync the main directory both ways — including sidecar files.

Only further research into the topic led me to the right solution, one that covered not just notes and highlights sync, but also reading progress. That solution turned out to be the koshelf project. Beyond those features, it also provides many others, including a pleasant web interface. Importantly, it can be run in several different ways, including from a binary file, which is key when using shared hosting. The whole process is well documented in the project’s documentation. The piece that ties koshelf and the device together is Syncthing, which needs to be configured on both devices: on the server (using the appropriate binary) and on KOReader (using the koreader-syncthing plugin). Given the functionality it provides, I decided to abandon phpkoreadersync in favor of koshelf, which gave me all the features I needed in one place. I made one more change — I decided to host Koshelf on my home server rather than on shared hosting, which first required setting up a reverse proxy (but that’s a topic for a separate post). The installation and configuration process went smoothly, as did setting up Syncthing. After a month of use I can recommend this solution — so far it has been reliable and meets my needs.