Sometimes doing things carefully isn’t so good

There’s been a problem in HourGlass regarding starting and stopping the playback of the fragment engine and the sound source previews when either one is already turned on. Basically the problems were because for whatever purposes, now lost to me, I wanted to turn off the audio hardware engine object whenever nothing was playing in HourGlass and turn it back on when some sound is to be played. It probably was some concern that keeping the audio engine on constantly wouldn’t be good for performance and so on. All this required keeping track of all kinds of state variables, which so far hasn’t ever worked correctly.

I however have now changed the behavior so that the audio hardware is kept constantly streaming and the needed audio playback objects are attached to it and removed from it as needed. That reduced a lot the need to keep track of all kinds of states about what is playing or not etc…This might potentially cause some problems with audio hardwares that don’t like to be shared between multiple applications,  so I might need to think this a bit further. (I’ve suffered audio hardware before *cough*DigiDesign*cough* that doesn’t want to be shared between applications, so I know it’s an annoying problem.) However, my current Focusrite USB interface doesn’t seem to mind this arrangement at all, even when using ASIO. I can have both HourGlass and Reaper, using ASIO, as well as WASAPI using applications open and playing audio, all simultaneously.

I don’t yet have any release date for an alpha or beta of HourGlass 2 at the moment. Lots of things to still do…

Posted in Uncategorized | Leave a comment

A writing on copy-on-write

Like they say at Reddit – tl;dr : HourGlass will get a better undo/version history.

The undo system for the envelopes in HourGlass has worked in a pretty odd way. All the envelopes (by default over 1000 envelopes) were stored for each undo history entry as arrays of bytes, even if an envelope had not changed at all compared to its state in the previous undo history entry. Now, this obviously sounds like a ton of data would be generated, most of it completely redundant, and no surprise really, that’s exactly how it works. A quite zany optimization  was thus introduced : the multiple byte arrays for the envelopes were joined into a single byte array which was then compressed using the zip compression algorithm. Yes, the same algorithm that is used for .zip files : http://en.wikipedia.org/wiki/Zip_(file_format) This in fact worked somewhat well to reduce the space needed for each undo step and even the CPU cost of zipping and unzipping the undo states wasn’t really that bad. (In reality the situation was a bit more complicated since map containers were also involved to associate the byte arrays with the parameters from which they originated, but the general concept is that the undo history was a dynamic array of dynamic arrays consisting of zip compressed data.)

The way the undo system worked has nagged me quite a lot over the years and I recently finally decided to re-implement it. I have investigated copy-on-write techniques quite a lot for the past year and decided to use that for the HourGlass envelopes undo history. Or rather, maybe really “version history” from now on. I don’t think the ability to record changes to an application’s state and getting them back should necessarily carry the rather sinister name “undo” as also creative/productive uses can be found. (For A/B/C/D… comparisons and so on. One could perhaps argue that such things should be implemented as separate features, but I happen to disagree with that somewhat.)

Copy-on-write means a way of managing data so that making actual copies is delayed until a copy of the data actually needs to change.

For example, an operating system can implement a scheme where it has a single page of memory that only consists of zeros which it can hand out to processes when they initially request memory to be allocated. (It improves security to give processes memory that is filled with zeros instead of some left-over contents from other processes. This unfortunately doesn’t mean that in programming languages like C or C++ things will be automatically zero initialized, but that is another issue.) Now, this zero page can be implemented so that it is initially shared among all the processes that have requested memory. Once a process wants to actually write into the memory it has allocated, the operating system switches the shared zero page to an actual reserved memory area. (Because the operating system is all powerful in the computer, this can all happen “under the feet” of the user processes.) While this sounds a bit complicated and useless at first, it might not be that uncommon that a process never actually writes into all the memory it has requested and thus making actual memory allocations could be avoided. In this case, copy-on-write has achieved a security improvement and a potential physical memory use optimization. (The “copy” in this scheme is the zeros that the process will initially see regardless of if it has yet written to the memory or not. If it just reads the memory, it sees the shared zeros. If it does a write operation, it will now see zeros in a new actually reserved memory area.)

The previous explanation is about a rather low level thing in the operating system but user level applications can also leverage the same technique. The traditional use case (in C++) has been to optimize string objects. (edit : code snippet put to pastebin.com)

http://pastebin.com/R6tZvm4j

One might wonder in the previous examples how can the first = assignment not make a real copy, yet the second will cause something different. The short answer is “magic”. The long one is that different overloads of C++ copy constructors and copy assignment operators have been invoked here by the compiler and those different overloads either just increase the data buffer reference count or “detach” the data buffer. If this sounds like the “wrong” overload might be picked sometimes and copies be created when not really needed, that indeed is the case if the programmer isn’t quite careful. (Mostly in observing const-correctness.) We can see that implementing a copy-on-write string class with an API that is natural to use and really does copies just on writes has problems. But wait, it gets worse…

The reference count to the shared data in a copy-on-write data structure can’t realistically these days be just a normal integer variable. The reason for that is that code will now very often run in a multithreaded context. The reference counting must be implemented in a way that is safe for multiple threads to use concurrently. Even if some particular piece of code currently doesn’t even use multiple threads, the reference counting should still support multiple threads properly. (Because maybe in the future the code will actually run in multiple threads and if someone forgot the reference counting in some piece of library code wasn’t thread-safe, only disaster and hilarity can ensue…) The most commonly used solution for the reference counter is to use an atomic variable.

Copy-on-write is not a popular thing for string classes now, the greatest reason being the atomic reference counting inside. The atomic reference counter can in some scenarios cause multiple threads to just wait on each other to synchronize, which can be a huge performance problem on multiple CPU systems. Saving on memory use and ensuring copying the strings is fast has therefore caused using multiple threads to be in vain in order to increase application throughput. It would have been better if each thread just used its own full copy of the string involved. I am myself also of the opinion that copy-on-write for strings probably isn’t a good idea anymore. However, Qt’s QString and gcc’s C++ standard library std::string still use it, although the problems the copy-on-write behavior was originally intended to solve don’t largely exist anymore. (Due to move semantics in C++11 and compilers that have improved to optimize copies away in the most obvious cases.)

So, getting back to HourGlass’s undo history…If copy-on-write has such suspicious properties in terms of it actually managing to avoid copies and memory allocations, and causing problems in multi-threaded environments, why would I choose to use the technique? Firstly, the data to be handled in HourGlass isn’t strings. Secondly, I have been using a copy-on-write wrapper class that requires on use of the contained object to explicitly determine if a read or write is going to happen. This removed the ambiguities with the copy constructors and copy assignment operators, as well as usually enforces const correctness. There is some added noise in the code due to the read() and write() calls, but it’s a trade-off I am willing to live with.

Usage is like (edit : code snippet put to pastebin.com because the WordPress code formatting doesn’t like C++ templates…) :

http://pastebin.com/b9sMRBm0

Almost all use of the envelopes in HourGlass was changed to be via the read() and write() methods. This took me a few hours and wasn’t exactly “fun”, but I think the results are worth it. The undo (or version) history for the envelopes is now a lot cleaner, no tricks with byte arrays and zip compression are needed and possibilities for various nice tricks now exist. For example, the state of just a single envelope could be easily and cleanly retrieved back from the history. Memory usage of the history entries has dropped to at least 1/3 of the old system. Adding more state besides the automation envelopes, like effects settings, has become a more plausible option too. I am somewhat on the fence about the effects settings though. I haven’t always found it useful for example in Cockos Reaper that all tweaks in plugins (or the mixer) go into its undo history. However, the way the state versions are now handled in HourGlass, it should be fairly trivial to implement it so that things are always stored, but can be optionally recalled from the history. Unneeded states can also be removed from the history by user action.

The problem with the atomic reference counters is mostly negated by HourGlass having to synchronize access to the objects anyway via mutex locking because I allow changing the states during audio playback. (That is, whatever performance decrease there exists due to thread synchronization, already would exist in any case.) It could be argued that the mutex locking isn’t appropriate to begin with in a realtime audio playback context, but so far I really can’t say I would have seen any really ill effects out of it, barring some obvious programming errors. I have been on the side searching and fixing issues in the code where the mutex locks could be held too long and replaced them with much faster code that has predictable characteristics.

Thanks to Sean Parent for his talks, materials and emails on copy-on-write (and on other stuff). Apologies for any errors, omissions and inconsistencies.

 

Posted in Uncategorized | Leave a comment

HourGlass 2.0 development started

I am not sure when there will be a first alpha/beta release, as there really aren’t any major exciting new additions done yet. The first thing I am looking into is cleaning up and optimizing the undo history with a copy-on-write system. (This hopefully reduces memory usage and allows richer things to be done with the undo history, which I will probably name to “version history”.) The second thing I will probably work on is replacing the PortAudio library with a new audio hardware access library I have been jointly developing with Vesa Norilo. Neither of these changes will likely result immediately in any big user experience changes.

I have replaced the Visual Studio 2010 compiler on Windows with Visual Studio 2013, which makes supporting Windows XP such an additional pain that I am likely not going to do it and the minimum OS requirement will be Windows 7. So, sorry Windows XP users…(What, there was an OS called “Vista”…? ;) )

I will of course still keep the 1.4.x code base around and I will make one or two releases out of that, that will still run on Windows XP.

If there will be a 1.4.x release for OS-X, that will however require at least OS-X 10.7, as will the 2.x releases of course.

Hopefully these news didn’t disappoint people stuck with older operating systems too much. But progress must go on.

Posted in Uncategorized | 2 Comments

HourGlass 1.4.5 released

Changes :

  • Preliminary feature to allow envelopes to play at different rates, as a multiplier of the texture length. (ie, if the texture length is 8 seconds, setting an envelope rate of 8 will make that envelope play 8 times during the whole texture.) The dial allows changing the speed to some common multiples of the texture length, the text box can be used to enter a free rate multiplier between 0.001 and 1024. This feature currently conflicts with the looped texture playback. The workaround is to disable the looped playback. (The sounding result will however effectively still be looped for most envelopes.) Also, when offline rendering, longer progressions of multiple playback rates might not evolve as far as desired, as the offline rendering ends at the texture length plus the tail length. The workaround is to use the live output recording feature. I will try to come up with some solutions to these in the future.
  • Maximum voice count increased from 256 to 512. It will depend on the computer capabilities, the used resampler mode and use of multithreaded voice processing if all the voices can actually play.
  • While making the above change, I noticed a performance problem with high voice counts, which hopefully is now fixed.
  • Sound volume to envelope converter window length now has name and value labels in the GUI
  • The offline chain processing progress indicator would sometimes get stuck over the waveform view, hopefully now fixed

Download :

www.xenakios.liitin.net/public_html/HourGlass_1.4.5.zip

Posted in Uncategorized | 2 Comments

HourGlass 1.4.4 released

Changes :

  • Source sound volume can be analyzed and converted into an automation envelope (found under the Ops menu button as “Generate from sound”, the unnamed slider changes the window length of the analysis, I’ll label etc this in a future update)
  • Source sound pitch can be analyzed and converted into an automation envelope.(*) Don’t expect too much out of this, it’s mainly useful to get an envelope that vaguely resembles the pitch curve of the source sound. (found under the Ops menu button as “Generate from pitch analysis”)
  • Fragment scripts can have 2 different sections that are called at seek time or at fragment generation time (forgot to include an example of this in the script presets…) (**)
  • When used as an external editor, no longer require a render target folder to have been set
  • Effect/VST plugin reported latency now taken into account in the offline preprocessing (ie, the assumably silent beginning of the processing is removed in the processed result)
  • Offline preprocessing resets effect/VST before rendering (should fix for example ReaFIR losing the start of the sound)
  • Envelope generation function rewritten so that the audio engine mutex lock is held only for a very brief time, even if tons of envelope points are generated
  • Some other changes/fixes

Download (Windows-only, OS-X builds still on indefinite hold) :

http://www.xenakios.liitin.net/public_html/HourGlass_1.4.4.zip

(*) The pitch analysis uses the csound “plltrack” opcode source code, which is LGPL licensed. So I added some plugin APIs for HourGlass to allow dynamically loading the pitch analysis binary. The source code for the pitch tracking plugin and the modified csound opcode is also included in this release. If you want to modify/rebuild the plugin yourself, you must use the Microsoft Visual Studio 2010 C++ compiler with Service Pack 1. Note that the plugin APIs and the required compiler will likely change in future releases of HourGlass, requiring to do rebuilds of the plugin!

(**) Example script that does a random walk for the source position fragment parameter, starting the random walk at the middle of the source sound play range when the HourGlass playback is seeked. If the source position would go before the beginning of the play range, it jumps to the end of the play range and vice versa.

@seek
rw=0.5;

@fragment
delta=-0.025+0.05*rand(1);
offset=rw;
rw+=delta;
rw < 0.0 ? (rw=1.0;);
rw > 1.0 ? (rw=0.0;);
Posted in Uncategorized | 7 Comments

HourGlass, from audio to envelope

This is a rather obvious feature that I should have done a long time ago. It analyses the source sound envelope shape and then feeds that into the shaping envelope and the target envelope is then generated out of the shaped values. It is mostly useful when used during the Play Forward mode, to keep the generated envelopes in sync with the source sound, but can of course be used in other ways too, if so desired.

Posted in Uncategorized | 3 Comments

C++11 : The three dots, that is variadic templates, part …

Variadic templates in C++11 allow writing functions that take in multiple parameters that can be of varying types.(*) What could this be useful for?

We sometimes have to be able to generate hash values out of objects, that is a number that represents that object uniquely. (Introductory article : http://en.wikipedia.org/wiki/Hash_function ) If even one bit in the object changes, we hope to see a “big change” in the generated hash value. The details of generating hashes is beyond the scope of this blog post and I will just use the hash facilities available in the C++11 standard library as well as a snippet borrowed from the Boost library. The quality of the hashes produced by the presented code may be sub-optimal but this blog post mainly attempts to demonstrate the C++11 variadic templates, not ideal algorithms to generate hash values.

Let’s say we have a class like :

class hash_test_object
{
public:
    double x=0.0;
    double y=0.0;
    double z=0.0;
    std::string id="Unknown";
};

How can we get a hash value out of an object instance of that? The C++11 standard library provides hash functions for some basic types, including double and std::string that we have in the class. It does not however provide anything to help us to get a hash from multiple values of those basic types. It is possible, however, to write a function like :

template<typename T>
void combine_hash(size_t& seed, const T& x)
{
    // Borrowed from Boost
    seed ^= hash_helper(x) + 0x9e3779b9 + (seed<<6) + (seed>>2);
}

Which calls the suitable overload of hash_helper :

template<typename T>
size_t hash_helper(const T& x)
{
    return std::hash<T>()(x);
}
// Dirty trick to enable hashing string literals, may be unacceptably slow in certain situations
size_t hash_helper(const char* x)
{
    return std::hash<std::string>()(std::string(x));
}

Then we can do the following to get a hash value out of the object :

void do_multicall_hash_combine()
{
    hash_test_object obj;
    size_t the_hash=0;
    combine_hash(the_hash,obj.x);
    combine_hash(the_hash,obj.y);
    combine_hash(the_hash,obj.z);
    combine_hash(the_hash,obj.id);
    std::cout << "(multicall) hash is " << the_hash << "\n";
}

Ok, so this works and we could call it a day. Or we could get annoyed about having to write that kind of code. If only there was a way to just do the hash combining with a single function call…And there is, by using variadic templates. I’ll just go straight to it.

template<typename T>
void combine_hashes_helper(size_t& seed, const T& x)
{
    seed ^= hash_helper(x) + 0x9e3779b9 + (seed<<6) + (seed>>2);
}

template<typename T, typename... Args>
void combine_hashes_helper(size_t& seed, const T& x, Args&&... args)
{
    seed ^= hash_helper(x) + 0x9e3779b9 + (seed<<6) + (seed>>2);
    combine_hashes_helper(seed,std::forward(args)...);
}

template<typename... Args>
size_t combine_hashes(Args&&... args)
{
    size_t seed=0;
    combine_hashes_helper(seed,std::forward(args)...);
    return seed;
}

That sure is painful to look at, but hopefully writing that kind of code will be the responsibility of library code developers. The client code that wants to use that code will be a lot simpler. (One could of course simultaneously be both the library and client code developer. It does require somewhat of a split personality…)

void do_variadic_hash_combine()
{
    hash_test_object obj;
    size_t the_hash=combine_hashes(obj.x,obj.y,obj.z,obj.id);
    std::cout << "(variadic) hash is " << the_hash << "\n";
}

A lot cleaner on the client code side compared to making the multiple calls to the hash combining function! Obviously it would be even nicer if we had some way to enumerate the class’s member variables automatically and make the hash out of those, but since C++ sadly still doesn’t have such reflection capabilities, we will have to settle to having to write out the member variables of interest ourselves. (We could of course develop a tool using for example Clang that would autogenerate this kind of stuff from the results of static code analysis, but that has some downsides and difficulties too.)

Additional examples of calls into the variadic function :

// The last zero being either float or double changes the hash,
// which is what we want
combine_hashes(0.0,0.0,0.0f); 
combine_hashes(0.0,0.0,0.0);
combine_hashes("The black cat walked on the street",42,0.0001,true);
combine_hashes("The black cat walked on the street",42,0.0001,false);
combine_hashes("The white cat walked on the street",42,0.0001,false);
combine_hashes("The white cat walked on the street",42,0.0002,false);
// Same numbers but in different order 
// give a different hash, as they should
combine_hashes(100,101);
combine_hashes(101,100);

I didn’t go into the details of writing the variadic function templates here, why the things need to be like they are in the library code and so on. I just wanted to give one hopefully motivating example where the C++11 variadic templates can be used to simplify the (client side) code. There are of course variadic class templates too… ;)

Related video of interest :

http://channel9.msdn.com/Events/GoingNative/GoingNative-2012/Variadic-Templates-are-Funadic

(*) The C language of course has had for a long time the capability to do something vaguely similar, variadic functions, the most familiar example being the printf() function. The C variadic functions are a mess though and since we live in the year 2014, nobody should really be writing C anyway unless absolutely necessary.

Posted in C++, Programming | Leave a comment