Category Archives: Programming

Unity: Reporting UI progress properly

UnityEditorUtility.DisplayCancableProgressBar can be used to report progress of an operation, but updating the progressbar itself comes with a performance overhead and can affect the performance of your application.

In this post, I’d like to share how I prefer to update a progressbar to minimize the time spent on the UI. It’s so simple, that I was even uncertain if it’s worth a post, but since I see every now and then Unity extensions that report progress inefficiently imo, I decided to put it online. It’s also related to my previous posts (1 2 3), where I describe what I changed to make my Unity plugins perform faster.

Let’s assume we have x tasks we need to complete and want to report the progress. The simplest, but also naive way to do this is something like this:

Unity EditorUtility.DisplayCancableProgressBar

If calling DoWorkOfOneTask() is particularly expensive, the overhead of updating the progressbar every loop-iteration is to neglect. However, if DoWorkOfOneTask() is fast, you most likely end up spending more time updating the UI than “doing the work”.

Let’s take the following editor script example, which you can copy&paste and run in Unity yourself:

Both methods basically do the same, except one reports the progress while the other does not.

Executing NoProgressbar() takes about 1 millisecond to complete, whereas WithProgressbar() requires more than 10 seconds. These 10 seconds are spent just on updating the progressbar UI. It’s sort of artificially made slower due to the progressbar.

It’s often not known in advance how long a certain operation takes to complete. For example, if DoWorkOfOneTask() uses some sort of caching, the first few calls might be slow, but subsequent calls could be much faster. So it does make sense to display progress when the operation takes a bit longer, but makes no sense when it would be finished instantly.

What I find useful is to update the UI in timed intervals, only after a certain amount of time elapsed. This ensures fast operations stay fast, because there is either no or at least not much overhead of updating the UI and for slow operations, updating the UI isn’t an issue anyway.

Here is my personal choice how I like to report progress in my applications:

The code above displays a progressbar only, when the total time of operations takes longer than 1 second and then updates the UI in 100 ms intervals at most. This assumes each of the DoWorkOfOneTask() calls isn’t particulary expensive. If such call takes eg 1 minute to complete, the progressbar wouldn’t be displayed until the first DoWorkOfOneTask() completed.

Reporting progress like this works like a charm in my applications, such as Texture Overview and AudioClip Explorer. If their cache is up-to-date, starting the plugin is so fast that no progressbar is necessary at all. However, if a few cache entries are outdated and updating them takes a while, the progressbar appears after 1 sec and gets updated periodically until it completes.

Unity: Reading assetTimestamp’s blazingly fast

UnityIn my previous posts (1 2), I was complaining reading the timestamp of an asset using the “AssetImporter.GetAtPath().assetTimestamp” API can be enormously slow.

In this post, I describe what I came up with to read thousands of timestamps in a few milliseconds only, rather than waiting an hour when using the official Unity API.

Upcoming versions of my Unity plugins feature a cache file that holds asset properties which are expensive to read. Once these properties have been read, they are stored in the cache file, so the next time they are requested, they can be served much faster.

The problem I encountered was to detect if an asset has been modified, to refresh the corresponding cache entry, in an acceptable period. Something like the following pseudo-code with 3000 textures stored on a 7 years old harddrive of mine took about 20 minutes to complete:

The issue, as explained in my previous post, is that “AssetImporter.GetAtPath” loads the asset into memory. Thus, the time the call requires to complete highly depends on the asset filesize itself.

While digging in various Unity related project folders to find a workaround, I came across the file “Library/assetDatabase3”. This file stores timestamps, along other data, of all assets in the project.

The format of assetDatabase3 changed from Unity 3.x to Unity 4 and it’s very likely that it can change in a future release too. Thus, it’s important for the plugin not to break when reading assetDatabase3 fails.

While reading this file, Unity shouldn’t updating it either. Running the code from the main-thread will do the trick, since it blocks the application and Unity won’t start to import new assets during this time.

In Unity 4, assetDatabase3 holds at offset 0x0C the offset where the timestamp table is located. In the following image, it tells us it’s located at offset 00 00 6B 80:

assetDatabase3 in HEX-Editor

Seeking at offset 0x6b80 in the same file, gets us at the at the start of the timestamps table:

assetDatabase3 in HEX-Editor

The timetamp table starts with:

For every element it is:

Timestamps need a some extra bit-magic to shift them into the correct order. The necessary bit-magic is also different for Mac and Windows. The final assetTime is basically the greater value of the two timestamps.

Reading assetDatabase3 and extracting all the assetTimestamps takes only a few milliseconds to complete. Once read, the plugin compares each of the timestamps to the cached data entries. If they are different, the cache entry is assumed to be invalid and will be refreshed. If reading assetDatabase3 fails, the plugin handles it as all cache entries are invalid.

This change improves the startup time of my plugins dramatically. I tested it with several projects and different Unity 4.x versions and it doesn’t really matter how many assets they contain. When opening the plugin, such as Texture Overview, it’s ready in about 0.5-2 seconds always (assuming the cache file is uptodate and assets have not changed/reimported). Most of these projects contain about 3000 textures and 1000 audio clips. One of my artificial test project contains 20000 audio clips. The plugin opens and is ready-to-use in all projects almost instantly.

VsTortoise, source code available at codeplex.com

I decided to release the source code of my Visual Studio add-in VsTortoise on vstortoise.codeplex.com, where all VsTortoise related files can be found right now, such as the C# source code but also the inno-setup installer script.

VsTortoise is an add-in for Microsoft Visual Studio, that provides an interface to perform the most common revision control operations directly from inside the IDE. It uses TortoiseSVN to execute the Subversion commands.

Nintendo DS: Sine approximation using fixed point math

When I was writing the Nintendo DS 4k intro, I obviously needed sin and cos functions to set up rotation matrices. Since using the C standard library for this task would have exceeded the 4kb limit easily, I was looking for an alternative that does not use much memory.

I was searching for an approximation method using taylor series and google made me visit these sites:

The only task left for me was porting it from float to fixed point. I didn’t really care about precision or performance much, something that reminds to a sine-wave and isn’t horribly slow would suffice.

Suprisingly, the fixed-point sin approximation is quite precise. At least more precise than I though it would be. Here is a screenshot of the original sinf routine (green) and the approximated one (red) using the source code below:

Nintendo DS sine approximation

Well, there is an issue with my code. When the incoming radians parameter is getting bigger (1000 and above), precision is getting worse, but this is an issue that can be fixed if you’re up to.

The routine from polygon labs can’t really handle radians greater than 2*PI, which I tried to fix by normalizing it to an angle between -1..+1 and then just use a bit-AND to discard all numbers outside this range.

I wanted to keep the radians format for the incoming parameter, because it’s how the standard sinf routine works. Using another range, like 0..4096 to represent a full rotation would probably solve that precision problem with large radians because this would remove the need for the first two code lines, but I want 2*PI.

Here is the source code, to be compiled with libnds and devkitARM:

Let me know when you fixed the precision problem with large radian values 🙂

Edit: Jasper “cearn” Vijn apparently likes challenges or in his own words:

So here I am, looking forward to a nice quiet weekend; hang back, watch some telly and maybe read a bit – but NNnnneeeEEEEEUUUuuuuuuuu!! Someone had to write an interesting article about sine approximation. With a challenge at the end. And using an inefficient kind of approximation. And so now, instead of just relaxing, I have to spend my entire weekend and most of the week figuring out a better way of doing it. I hate it when this happens >_<.

Don’t miss to read his article Another fast fixed-point sine approximation.

For completeness, I post a fixed version of the method above. I highly encourage you to visit Jasper’s site and get one of his better sine approximation functions though.

 

World’s first Nintendo DS 4k Intro

During eastern 2009 I followed the breakpoint demo party via live stream and was especially attired in the 4k intro competition, which motivated me to try if it is possible to create an application that draws a simple quad in 4096 bytes on the Nintendo DS, using the devkitPro tool-chain.

But even compiling a simple c source file containing only the application entry point, without any external libraries, creates an output file that is 54848 bytes large:

After fiddling around for some time, I had a Nintendo DS application that not only shows a simple quad, but hundreds of lit textured cubes, creating a tunnel where the camera flys through as well as a fullscreen distortion effect in less than 4096 bytes.

Once released on pouet.net, I figured from the comments I could had shrink the filesize further 480 bytes by just compressing it. I feel so dumb!

It obviously can not compete with the breakpoint 4k entries, but it was fun to do it anyway, especially because it is the world’s first 4k intro for the Nintendo DS! Unfortunalety, it features no music.

You can download the binary at: http://pouet.net/prod.php?which=53081