A recent use case required me to move an entire repository to another repository while preserving history. A simple git subtree would do the move, but one little problem is after the commit history moved over, the commit messages would have references to the pull request number from the original repo, then Github would create a link using the PR number but based on the new repo, which would point to a wrong PR in the new repo.

To avoid that, I'd need to rewrite commit history. But since there were many many commits, I had to figure out doing this in a scriptable way. This is where git filter-repo comes in handy, which is the replacement of the old filter-branch.

To do that, filter-repo has a filter for commit messages. The filter also has a --message-callback option, which would be followed by a Python code body to process commit message. The callback would give you the original commit message as the standard input and use the output as the new commit message, both in byte strings. In this case, a simple regular expression would do the trick:

git filter-repo --message-callback 'return re.sub(br"(#(\d{1,3}))\n", br"(https://github.com/FooOrg/BarRepo/pull/\1)", message)'

Another consideration is I only wanted to rewrite the commits from the original repo, but filter-repo by default tries to rewrite all the commits in the repo. --refsargument is exactly for this purpose where you can specify a range of commits. For example, I have all the commits in my local branch ahe/foo, then I will just tell filter-repo to only rewrite commit messages for the diffs between the main branch and ahe/foo branch: --refs main..ahe/foo.

Just a little background, the instance is running on Ubuntu 18.04 LTS at DigitalOcean.

The first step is to make sure npm and nodejs is updated to the latest version supported by Ghost. Go to https://ghost.org/faq/node-versions/ to find the recommneded nodejs version.

# update npm to the latest
npm install -g npm@latest

# update nodejs to recommended version, e.g., v12
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs

Once you have npm and nodejs updated, run sudo npm install -g ghost-cli@latest to update to the latest Ghost CLI.

Next, go to the directory where the instance is running, then switch to the user who owns the directory.

Be sure to check the Upgrading Ghost page to find out if additional instruction is needed, especially when there's a major version upgrade.

Always back up the content directory! A simple cp -r content ~/backup-dir would do the trick.

Finally, it's time to run ghost update to actually update Ghost.

Well, I lied. ghost update sometimes may not be the final step. In case of failures, try running ghost update --force to do the update for the 2nd time, which might just work the 2nd time. If the update is successful, but the blog still isn't up and running, reboot.

If the update still fails, ghost doctor might give you some information. If you lose the patience by now, ghost update --rollback could be handy to revert back to the previous stable version of Ghost.

Lastly, if still nothing works, and you still want to proceed with the update and you don't want to spend too much time dealing with it anymore, just reinstall Ghost and restore from the backup.

You can hate WordPress however you want it, but it's never a pain to upgrade(if the environment is set up correctly).

Moshi 1.5 added @Json annotation support for Kotlin classes. It's a great addition for Kotlin projects and has been working really well for me. I'd just like to share three tips when working with Moshi Kotlin.

No generics support yet

Resovling generics types is still working in progress. AutoValue and its Moshi extension is the best alternative if you have such a need.

ProGuard

If it's unfortunate that your project has to use ProGuard in the release apk, there might be times a misconfiguration will surprise you, like some fields just gone missing. There are already many issues opened with people posting different workarounds. The rule stated in the README has worked for me until I bumped up Kotlin version to 1.50. And by the end of the day, we basically had to keep everything from kotlin.Metadata and all the model classes. :(

-keep class kotlin.Metadata { *; }
-keep class org.aaronhe.api.** { *; }

And if you are using Kotlin reflect 1.2, there's little extra configraution you may need to add: -keep public class kotlin.reflect.jvm.internal.impl.builtins.* { public *; }.

Adapter creation is expensive(for the first time)

Moshi Kotlin has a depdency on kotlin-reflect because it needs some extra information about the class, like what fields are being annotated, what's value of the @Json annotation etc. This transitive depedency will increase the apk size a little bit. If you are developing for Android, you might also need to be aware of when to call Moshi#adapter for kotlin classes. In order to get the information Moshi needs, Moshi has to get a hold of the primary constutor. However that's an expensive call because KClass#primaryConstructor will in-turn call getResourcesAsStream which could take 2 seconds on a Nexus 4 device running KitKat. Fortunately, Moshi caches generated adapters, so only the first time creation takes a performance hit.

I discovered this with the help from NimbleDroid. It was actually caused by a class having an adapter as its member variable and being injected at the application class. So the application startup time surffered from this. Making the adapter lazily initlaized was the easist fix.

There's also an interesting project called kotshi which uses annotation processing to generate Moshi adapters for Kotlin data classes. I haven't used this in production yet. But it looks very promising.

WebP is great, but transparency WebP is only supported natively on Android since 4.2.1. Even though everyone should have minSdk set to at least 19 by now, for apps that sill have a large amount of users on pre-4.2 devices, there's a little trick maybe apply to you for adding WebP "transparency" support.

The cwebp CLI provides an option called -blend_alpha which allows you to blend the alpha channel using a solid color. What that means is if you have a large PNG image with transparency area but always lays on top of a known background color, say white. Then, when you convert that PNG image into a WebP image, you can use -blend_alpha 0xffffff option to blend the alpha channel with white color. This will not only reduces the image size, but also works on pre-4.2 devices.

One of the nice things Kotlin provides is interoperability with Java. Existing Java code can just call Kotlin classes/methods as if they were Java classes/methods. However, when it comes to property getters, the equivalent Java method would have a get prefix. This is can be a little bit annoying.

Let's say we have this Experiment class has two getters started and ended:

class Experiment {
  val started get() = // fancy logic
  val ended get() = // another fancy logic
}

When calling those two getters from Java, it'd be experiment.getStarted() and experiment.getEnded(). The extra get prefix is a little redundant. There's a small trick we can do to avoid that is to use @JvmName annotation. Just annotate the property getter like this:

class Experiment {
  val started @JvmName("started") get() = // fancy logic
  val ended @JvmName("ended") get() = // another fancy logic
}

Then, in Java, we now are be able to just call experiment.started().

In Retrofit 2, when we need to add additional headers, we would just add an implementation of the Interceptor.

But sometimes, we only need to add a particular header for one API call, and the value is set dynamically at runtime. It's a bit tedious to add another interceptor just for that call.

The @Header annotation is really handy for this case. So we can pass a parameter annotated with @Header to the API call method. And Retrofit will take care of it.

Let's take look at an example:

@GET("/weather")
Observable<WeatherInfo> getWeather(@Header("foo") String value, Location location);

That would allow us to pass a String value as a request header for the getWeather API call. If the value is null, the foo header will just be omitted from the request.

Recently, I've run into OutOfMemoryError: PermGen space issue when building Gradle projects, especially with Kotlin.

A common way to fix that is to increase the heap size for Gradle JVM, like adding org.gradle.jvmargs=-XX:MaxPermSize=512m to grade.properties.

Actually, there's another simple way to fix the issue which is using JDK 8 because it eliminates the PermGen space. More details can be found at Java 8: From PermGen to Metaspace or this StackOverflow answer.

Notes to Android developers, even though JDK 8 hasn't been supported by Android, but you can still use JDK 8 to build Android projects as long as you set compatibility level to 6 or 7 depends on minSDKVersion. Then you will get benefits like what we mentioned earlier.

Often times, I need to clean up stuff after a subscriber unsubscribes. There's a handy methods just for that: Subscriptions.create(Action0). The parameter Action0 is a function passed in to do things after a subscriber unsubscribes.

For example, if a callback is set on an instance when a subscriber subscribes to a stream, so the instance can emit items whenever the callback gets called. After the Subscriber unsubscribes, we need to free the listener from the instance to avoid side effects.
Or if the observable is a network call, you may want to cancel it when a subscriber unsubscribes.

Let's look at some code.

public static Observable<Integer> textLengthChanges(final Text text) {
  return Observable.create(subscriber -> {
    Text.OnLengthChangedListener listener = length -> {
      if (!subscriber.isUnsubscribed()) {
        subscriber.onNext(length);
      }
    };

    text.setOnLengthChangedListener(listener);
  });
}

This textLengthChanges observable emits the length of the text whenever it's changed.

A subscriber at some point would unsubscribe to the stream after it has done the work. Then we need to release the Text.OnLengthChangedListener from the text instance. It can be done by simply adding:

...
subscriber.add(Subscriptions.create(() -> text.setOnLengthChangedListener(null)));
...

Under the hood, Subscriptions.create(Action0) creates a BooleanSubscription which checks if it's got unsubscribed and if so, it will invoke the Action0.

I must say the naming Subscriptions.create(Action0) isn't that intuitive to me (and someone agrees on that). But because Subscription is equivalent to Disposable in Rx.Net, the naming makes some sense. It's also worth mentioning that it seems RxJava 2.0 is going with Disposable instead of Subscription.

Last but not least, don't be confused with Observable.using. This operator is useful when you have full control over lifespan of an object, so you can do some cleanups of the object when the Observable terminates.

Lately we have to deal with Android 65k method limit, so enabling ProGuard to remove some unreferenced methods might be a good way to temporarily solve this problem.

Enable ProGuard
It's really easy to enable ProGuard. If using Maven, just add the following code in your pom file under Android Maven Plugin's configuration:

<proguard>
    <skip>false<skip>
    <config>proguard-project.txt</config>
</proguard>

proguard-project.txt is the configuration file for ProGuard. I'll explain it later. If using Gradle as your build system, just add following code in buildTypes¹：

runProguard true
proguardFiles file('proguard-project.txt')

After enabling ProGuard, whenever the project is built, ProGuard will generate several text files: dumpt.txt, mapping.txt, seeds.txt and usage.txt.

• dump.txt: Internal structure of all the classes in the final .apk.
• mapping.txt: If obfuscation is enabled( by default it is), this file will list the mapping between original classes/methods/fields name and obfuscated one.
• seeds.txt: Lists classes and members are not obfuscated.
• usage.txt: Lists all the code removed from the final .apk.

• Warning: can’t find superclass/interface/referenced class: Generally, this might be the first or the only warnings you would encounter. The reason why this warning happens is really simple: some jars are missing, so you need to specify the locations of those jars. But for an Android project, in most cases just use -dontwarn to turn off those warnings because during Android build process, it will automatically feed you all referenced jars.
• -dontobfuscate: I just don't feel the need for code obfuscation yet. And it will make debug a little bit harder. ProGuard will generate a mapping file between original class names and obfuscated ones. By using retrace tools, it will make life easire. I am sure there's an automated way to do that. For Crashlytics, it supports reading the mapping file, so you don't need the extra step to using the retace tool.
• Automated Instrumentation Testing: If the Android project has some instrumentation tests, -keep public class * extends android.app.Application is a must. There will be an class generated extends android.app.Application class for each test project but only referenced in AndroidManifest.xml, so ProGuard just thinks that Application class is unreferenced. Lost a whole day of productivity because of this. :-(
• Play Services: This gigantic library contains about 20k-30k methods. Even though it's generally considered a must-have in Android development, lots of classes/methods might just sit there doing nothing. Google has its own recommended configuration for this library. Just include those rules. Good news is, the next release of Play Services, v6.5, will be modularized.
• Reflection: Anything using reflection should be kept.

Multi-dex
Mobile app is really a thing now, there still might be a chance your project references more than 65536 methods even after ProGuard is enabled. Some guys come up with an idea: compile code to multiple dex files, one of those is the primary, custom loading dex files when needed. Well, before ART is released, the new runtime for Android, this is very cumbersome, inefficient and error prone. As ART is released alone with Lollipop, multi-dex gets native support. Multiple dex files will be compiled to one .oat file, executable for ART. For pre-ART Android, Google also has a support library for that. More details for setting it up could be found here: Configuring Your App for Multidex with Gradle. It seems solved the problem a bit nicer, but still there are several limitations, like doesn't work well for pre-4.0 devices, adds compiling overhead, etc.

The real problem for 65k looks very simple though. It might just be a really hard bug for Google to fix. On the other hand, mobile apps are doing more and more things, there should be a perfect solution for this.

1. There are other ways to configure ProGuard for Gradle. getDefaultProguardFile('proguard-android.txt'),'proguard-rules.pro' looks neat. First find the default configuration file from Android SDK then merge it with your project specific config. This is something Android Maven Plugin cannot do yet.