How’s HourGlass? Do I now drink JUCE? Qt no longer so cute?

I haven’t worked on HourGlass at all for a few months. Now that I’ve gotten more familiar with JUCE, it seems there could be a possibility I might rewrite HourGlass in some form using JUCE. That would have the benefit that making a plugin version would finally feasibly be possible. The recently released CDP front-end plugin is actually mostly developed as a standalone application (binaries of that have not been publicly available yet), which I every now and then build and test in the Reaper plugin mode. With JUCE based code it’s quite transparent to switch between a standalone application and a plugin with the same code base.

I am not sure yet what I am going to do with HourGlass. I feel very tempted to just drop the Qt based version completely. While Qt is an excellent framework, I’ve found it to become less and less relevant for myself. It’s very bad for doing plugins of any kind, except plugins for host applications that themselves are written with Qt. (And even then, there are the problems of C++ ABI compatibility and so on…) Qt no longer develops the traditional Qt Widgets module. They instead prefer to develop their Qml based GUI system, which isn’t very attractive for me. (Qml has a TON of runtime dependencies, it’s not C++ etc…) They nudge people towards using Qml in new code and I don’t like that.

One possibility how I might proceed is that I could extract and adapt the HourGlass audio processing code from the current Qt-based HourGlass code base and integrate that code into the CDP front-end code. The CDP programs offer all kinds of granular sound processing algorithms, but none of the CDP programs do granular processing exactly like HourGlass. (At least as far as I’ve found…) So having the HourGlass granulation/fragmentation processing available in the CDP front-end could make sense. I might need to rename the CDP front-end to something else then, though…

JUCE is not without its own problems, of course. But I’ve started to feel like it is the more appropriate thing for me to use.

I guess these were just random ramblings…

Posted in Uncategorized | 1 Comment

Oh noes, there could be a null pointer!

It’s been some time since I wrote anything about C++ programming, so here’s something. Sorry for users of HourGlass and other softwares of mine if this isn’t of much interest. ;) (I will probably start a separate blog specifically about programming later this year.)

Ideally one’s C++ code would never attempt to use an object via a null pointer, but in some cases having the possibility of a null pointer is just convenient to have. Maybe it’d be too tiresome to write an object that has some well defined null/empty/do-nothing state or whatever.

Class whose methods will be called in the examples :

class MyObject
{
public:
    double m_x=0.5;
    double foo() { m_x+=0.1; return sin(m_x)+cos(m_x); }
    void not_much() { }
    double borken_sum(double a, double b) { return m_x+a+b; }
    std::string mangle(std::string a)
    {
        std::reverse(a.begin(),a.end());
        return std::to_string(m_x)+a;
    }
};

The usual way to deal with a pointer that can be either null or valid is something like :

// MyObject* m_object ends up as nullptr or a valid pointer
// in some way before this code is run
// foo() of valid object should return a double
if (m_object!=nullptr)
  return m_object->foo(); 
else
  return 0.0; 

Or, more tersely with the terrible ternary operator :

return m_object!=nullptr ? m_object->foo() : 0.0;

I today came up(*) with an alternative solution that involves quite a hairy helper function but I kind of like this solution, assuming it actually works properly in most scenarios. (I didn’t extensively test this yet. There could be cases where the compiler gives an error or maybe the performance of making an object method call this way is horribly bad. Maybe Clang and GCC totally hate this. This was tested with the Visual Studio 2013 C++ compiler.)

The helper function in all its insanity :

template<typename U,typename F,typename... Args>
auto safe_call(U* ptr,F f,Args... args) -> decltype(std::mem_fn(f)(ptr,args...))
{
    if (ptr!=nullptr)
        return std::mem_fn(f)(ptr,args...);
    return decltype(std::mem_fn(f)(ptr,args...))();
}

Usage in client code :

return safe_call(m_object,&MyObject::foo);

In this version, no annoying additional control flow needs to be written, “nullptr” doesn’t need to be written, “m_object” doesn’t need to be written twice and the fallback value of 0.0 will be default constructed from the type the MyObject::foo() method returns.

Due to the variadic template, member functions with 1 or more arguments can also be used. The auto return type in the helper function also works for member functions that don’t return a value.

Additional usage examples :

double a=safe_call(m_object,&MyObject::borken_sum,2.0,1.0);
std::cout << safe_call(m_object,&MyObject::mangle,"Cat Attack") << "\n";
safe_call(m_object,&MyObject::not_much);

The downside is having to write the names of the class and the method with the &ClassName::MethodName notation.

Of course the safe_call-style technique doesn’t buy much if several methods of an object need to be called in a row. The following usual solution will work just fine and expresses the intent :

if (m_object!=nullptr)
{
  double a=m_object->foo();
  m_object->not_much();
  auto r=m_object->mangle("ooF");
} 

(*) No doubt this has been invented by someone else before me but I wanted to see if I could write it myself.

Posted in C++, Programming | Leave a comment

CDP frontend Reaper extension plugin

edit 29th July 2014 : pre2 builds uploaded. Contains various fixes and the ability to zoom and scroll the input sound waveform. (The zoomscroll-bar is between the input and output waveforms.)

I am finally making the first public release of the C++/Juce based CDP frontend I’ve worked on for the last 5 weeks or so. Thanks to Oli Larkin for working on the OS-X build! (Unfortunately it didn’t materialize for this first release, though.)

If you don’t already know what the Composer’s Desktop Project is, please read before trying to use the plugin to decide if this something you would likely be interested in trying to test :

http://www.composersdesktop.com/

To use this, you must have the Composer’s Desktop Project installed, available from :

http://www.unstablesound.net/cdp.html

-The plugin allows you to bypass setting the CDP_SOUND_EXT environment variable which is mentioned somewhere in the CDP documentation that it must be set for things to work. (If the environment variable is not set, the plugin sets it temporarily.)

-The plugin will on first run ask you to locate the CDP binaries location. This is the folder with files such as “pvoc.exe”, “modify.exe” and “distort.exe” within the CDP installation.

-The plugin adds one new action into Reaper : “Xenakios : Show/hide CDP front-end” which you should find from the Reaper actions window. The action is not added to any menu in Reaper or given a default keyboard shortcut.

-When the plugin window has focus, the following keyboard shortcuts are active :

Insert-key : Import the selected Reaper media item
Home-key : Toggle automatic rendering after settings (parameter sliders, automation envelopes, the time waveform time selection) have been altered. You might want to have this turned off when you know you want to alter several settings/automation points and the rendering delay is too short to reach the desired GUI elements before the automatic rendering is triggered.
Delete-key : Removes the selected area from the sound file or envelope points of the selected area. (Depending on whether the Waveform or Envelope button in the top left is active.)

-Please use this plugin with Reaper 4.7 or newer only! Reaper versions before 4.7 have a bug that crashes during the realtime preview.

The front end has various omissions, quirks and defects at this point, but I hope it will be somewhat useful already. Some of the things are quite tricky to implement ideally because everything works by running command line programs in the background, after all…But I hope I will be able to implement better solutions to some of the issues.

It only has a limited selection of the CDP processing programs available at this point, as some of them require much more complicated GUIs and logic than is currently implemented. (And some of the suitable programs I just haven’t added yet.)

Many of the CDP programs only process mono files. This has not been worked around yet by processing stereo files by splitting them into 2 mono files and then at the end joining them back to stereo. Currently if given a stereo file for the mono-only processes the front end will ask if you want to process the file as mono. If you do so, the stereo file is first downmixed to mono.

No OS-X builds are available now. However if you are an enterprising OS-X user, you can try building from source code (but I can’t offer much, if any, help for that). To comply with the Juce licensing model, the source code is GPL-licensed though not yet explicitly mentioned so in the repository (I will update the source code files soon to reflect that) :

https://bitbucket.org/xenakios/reaper-cdp-frontend-extension-plugin/src

64 bit plugin : (Windows 7 or newer) :

http://ge.tt/5XjI0fp1/v/0?c

32 bit plugin : (Windows 7 or newer) :

http://ge.tt/64MD1fp1/v/0?c

Posted in Uncategorized | Leave a comment

Python scripts for using CDP in Reaper

Again a little side project to distract myself from making progress with HourGlass 2…

If you don’t know what the Composer’s Desktop Project or Reaper is, this set of Python scripts to be run from Reaper probably isn’t something you want to try.

Prerequisites :

  • You have confirmed the CDP programs will run for you (ie, you have installed the CDP package, set the annoying environment variable for wav-files, done some test runs from the command line/SoundLoom/Soundshaper etc)
  • You have installed Python 3.x and set up Reaper to use it for ReaScript (Python 2.7.x might work too, but I haven’t tested)

Download the zip of the Python scripts :

http://ge.tt/60IhTti1/v/0?c

If ge.tt appears to be down, you can download the source codes as a zip archive from the Bitbucket repo :

https://bitbucket.org/xenakios/cdp-python-reascripts/get/c5316792a198.zip

There are 15 .py files. All except one (cdp_helpers.py) may be loaded into Reaper as new Python ReaScript actions. They allow you to process the selected Reaper media item with the CDP sound process in question.

There unfortunately is a set up step which for now has to be done manually by editing the cdp_helpers.py file. :( You must change the line with the text :


cdp_prog_dir=""

to something like


cdp_prog_dir="C:\\Portable_Apps\\cdpr700-InstallPC\\_cdp\\_cdprogs\\"

Where you would substitute the actual location of the CDP binaries on your machine inside the “”. (This will be the CDP installation directory with files such as blur.exe, pvoc.exe, envel.exe) Note that on Windows, each \ needs to be substituted with \\ instead.

In regards to the CDP processings themselves, you should refer to the CDP documentation. I’ve tried naming the script files so that the corresponding CDP processes can be found from the documentation without too much trouble. The scripts obviously don’t cover all the CDP processings available, more will be added in the future. The 14 scripts just provide some examples which I chose somewhat randomly or according to my own taste.

Processing obviously happens offline but is not in any way destructive, that is, the original audio files in the Reaper project are never modified in any way. The new processed file will be added as a new take into the Reaper media item. If the processing is supposed to change the length of the material, the item length is changed to correspond to the new length. (This might be nice to have optionally, I guess…?)

The first limitation at the moment is that several of the processings will only work with mono audio source files. So, for those processors, when you have a stereo file, you should change the selected item’s channel mode in Reaper to mono and then use the Reaper glue action before running those CDP processes. There is a possible hack around this, which I will probably add to the cdp_helper.py file later.

The second limitation is that the facilities of many of the CDP programs to allow time varying parameters (aka automation) are not available from these scripts. In principle, this could be enabled by parsing break point envelopes from the dialog box text fields. So for example something like [0.0,0.1,10.0,1.0] could be written in a text field to automate a parameter so that it starts at the value 0.1 at the time 0.0 and proceeds to value 1.0 at the time 10.0. This however seems terribly clunky and I am not sure if it would be worth it implementing it.

The third limitation is that no audio preview of the processings is available. This could in principle be implemented, but not really with Python and ReaScript. For now, the only way to know how the processing ends up sounding is to enter some parameter values and run the process. If you are not happy with the results, switch back to the original take in Reaper and rerun the same or different CDP process.

I am not sure if these work on OS-X or not…But I just updated the download link with an obvious fix for the cdp_helpers.py file.

If you do cool and/or useful changes/additions to the scripts, it would be appreciated if you would let me know of those so I could maybe add them to the source code repo, hosted at  https://bitbucket.org/xenakios/cdp-python-reascripts/overview

I’ll expand this blog post a bit later but hopefully these instructions are enough to allow trying these.

Posted in Uncategorized | 4 Comments

What’s been going on?

I’ve taken some leave from developing HourGlass and been updating/rewriting an old Reaper extension plugin (from around 2009) that does pitch and time processing of media items in Reaper. I was prompted to do this when a 64 bit build of the plugin was requested at the Reaper forum a few weeks ago. I had already done a 64 bit build  of the plugin quite a long time ago, but I very quickly noticed the code had all kinds of flaws and worse, serious bugs where invalid memory was accessed, threads were writing and reading the same objects at the same time without locking and so on. For some reason, the 32 bit build of the plugin had worked seemingly OK.

The update or rather rewrite has involved the following and more :

  • Removal of numerous global variables. (Still not completely eliminated as only one instance of the plugin can run anyway currently and removing the remaining global variables would be a bit labor intensive.)
  • Remove almost all explicit uses of C++ new and delete operators and use smart pointers and std::vectors instead. The old code had a very distasteful hodgepodge of std::vectors and allocating buffers via operator new[], usually forgetting to release them with delete[]. Consistency is sanity, all dynamic buffers are now std::vectors. I wonder why I hadn’t thrown a lonely malloc()/free() pair into the old code, too…
  • Object ownerships were introduced. For example, in the old code the automation envelopes were for some reason in a global std::vector. Allocated via pointers for no purpose, of course. In the new code the audio processors themselves create and manage the envelope objects. And the envelopes are managed just as values, not pointers. (ie, in the old code there was std::vector<envelope*> as a global variable, in the new code the DSP processor class has a member std::vector<envelope>.)
  • Although deep class hierarchies can be problematic, I introduced a new base class that handles common things for the 2 derived DSP processor classes (for resampled pitch processing and for independent pitch/time/formant processing). The GUI code no longer needs to do insane checks about which class and which object instance should be accessed. The deeper class hierarchy was clearly a better design here. Adding new processor types is around 100x more feasible now than it was in the old code.
  • State saving/loading to/from disk was added, handled with JSON. This required writing quite a lot of annoying boilerplate code, but on the other hand clarified what the object states really consist of.

The plugin is in a pretty good state at the moment, I suppose I will keep working on it for a week or two. Then I will hopefully return to working on HourGlass 2. (A public release of HG 2 is still far in the future. An optimistic estimate would be around July 2014.)

Posted in Uncategorized | 1 Comment

Sometimes doing things carefully isn’t so good

There’s been a problem in HourGlass regarding starting and stopping the playback of the fragment engine and the sound source previews when either one is already turned on. Basically the problems were because for whatever purposes, now lost to me, I wanted to turn off the audio hardware engine object whenever nothing was playing in HourGlass and turn it back on when some sound is to be played. It probably was some concern that keeping the audio engine on constantly wouldn’t be good for performance and so on. All this required keeping track of all kinds of state variables, which so far hasn’t ever worked correctly.

I however have now changed the behavior so that the audio hardware is kept constantly streaming and the needed audio playback objects are attached to it and removed from it as needed. That reduced a lot the need to keep track of all kinds of states about what is playing or not etc…This might potentially cause some problems with audio hardwares that don’t like to be shared between multiple applications,  so I might need to think this a bit further. (I’ve suffered audio hardware before *cough*DigiDesign*cough* that doesn’t want to be shared between applications, so I know it’s an annoying problem.) However, my current Focusrite USB interface doesn’t seem to mind this arrangement at all, even when using ASIO. I can have both HourGlass and Reaper, using ASIO, as well as WASAPI using applications open and playing audio, all simultaneously.

I don’t yet have any release date for an alpha or beta of HourGlass 2 at the moment. Lots of things to still do…

Posted in Uncategorized | Leave a comment

A writing on copy-on-write

Like they say at Reddit – tl;dr : HourGlass will get a better undo/version history.

The undo system for the envelopes in HourGlass has worked in a pretty odd way. All the envelopes (by default over 1000 envelopes) were stored for each undo history entry as arrays of bytes, even if an envelope had not changed at all compared to its state in the previous undo history entry. Now, this obviously sounds like a ton of data would be generated, most of it completely redundant, and no surprise really, that’s exactly how it works. A quite zany optimization  was thus introduced : the multiple byte arrays for the envelopes were joined into a single byte array which was then compressed using the zip compression algorithm. Yes, the same algorithm that is used for .zip files : http://en.wikipedia.org/wiki/Zip_(file_format) This in fact worked somewhat well to reduce the space needed for each undo step and even the CPU cost of zipping and unzipping the undo states wasn’t really that bad. (In reality the situation was a bit more complicated since map containers were also involved to associate the byte arrays with the parameters from which they originated, but the general concept is that the undo history was a dynamic array of dynamic arrays consisting of zip compressed data.)

The way the undo system worked has nagged me quite a lot over the years and I recently finally decided to re-implement it. I have investigated copy-on-write techniques quite a lot for the past year and decided to use that for the HourGlass envelopes undo history. Or rather, maybe really “version history” from now on. I don’t think the ability to record changes to an application’s state and getting them back should necessarily carry the rather sinister name “undo” as also creative/productive uses can be found. (For A/B/C/D… comparisons and so on. One could perhaps argue that such things should be implemented as separate features, but I happen to disagree with that somewhat.)

Copy-on-write means a way of managing data so that making actual copies is delayed until a copy of the data actually needs to change.

For example, an operating system can implement a scheme where it has a single page of memory that only consists of zeros which it can hand out to processes when they initially request memory to be allocated. (It improves security to give processes memory that is filled with zeros instead of some left-over contents from other processes. This unfortunately doesn’t mean that in programming languages like C or C++ things will be automatically zero initialized, but that is another issue.) Now, this zero page can be implemented so that it is initially shared among all the processes that have requested memory. Once a process wants to actually write into the memory it has allocated, the operating system switches the shared zero page to an actual reserved memory area. (Because the operating system is all powerful in the computer, this can all happen “under the feet” of the user processes.) While this sounds a bit complicated and useless at first, it might not be that uncommon that a process never actually writes into all the memory it has requested and thus making actual memory allocations could be avoided. In this case, copy-on-write has achieved a security improvement and a potential physical memory use optimization. (The “copy” in this scheme is the zeros that the process will initially see regardless of if it has yet written to the memory or not. If it just reads the memory, it sees the shared zeros. If it does a write operation, it will now see zeros in a new actually reserved memory area.)

The previous explanation is about a rather low level thing in the operating system but user level applications can also leverage the same technique. The traditional use case (in C++) has been to optimize string objects. (edit : code snippet put to pastebin.com)

http://pastebin.com/R6tZvm4j

One might wonder in the previous examples how can the first = assignment not make a real copy, yet the second will cause something different. The short answer is “magic”. The long one is that different overloads of C++ copy constructors and copy assignment operators have been invoked here by the compiler and those different overloads either just increase the data buffer reference count or “detach” the data buffer. If this sounds like the “wrong” overload might be picked sometimes and copies be created when not really needed, that indeed is the case if the programmer isn’t quite careful. (Mostly in observing const-correctness.) We can see that implementing a copy-on-write string class with an API that is natural to use and really does copies just on writes has problems. But wait, it gets worse…

The reference count to the shared data in a copy-on-write data structure can’t realistically these days be just a normal integer variable. The reason for that is that code will now very often run in a multithreaded context. The reference counting must be implemented in a way that is safe for multiple threads to use concurrently. Even if some particular piece of code currently doesn’t even use multiple threads, the reference counting should still support multiple threads properly. (Because maybe in the future the code will actually run in multiple threads and if someone forgot the reference counting in some piece of library code wasn’t thread-safe, only disaster and hilarity can ensue…) The most commonly used solution for the reference counter is to use an atomic variable.

Copy-on-write is not a popular thing for string classes now, the greatest reason being the atomic reference counting inside. The atomic reference counter can in some scenarios cause multiple threads to just wait on each other to synchronize, which can be a huge performance problem on multiple CPU systems. Saving on memory use and ensuring copying the strings is fast has therefore caused using multiple threads to be in vain in order to increase application throughput. It would have been better if each thread just used its own full copy of the string involved. I am myself also of the opinion that copy-on-write for strings probably isn’t a good idea anymore. However, Qt’s QString and gcc’s C++ standard library std::string still use it, although the problems the copy-on-write behavior was originally intended to solve don’t largely exist anymore. (Due to move semantics in C++11 and compilers that have improved to optimize copies away in the most obvious cases.)

So, getting back to HourGlass’s undo history…If copy-on-write has such suspicious properties in terms of it actually managing to avoid copies and memory allocations, and causing problems in multi-threaded environments, why would I choose to use the technique? Firstly, the data to be handled in HourGlass isn’t strings. Secondly, I have been using a copy-on-write wrapper class that requires on use of the contained object to explicitly determine if a read or write is going to happen. This removed the ambiguities with the copy constructors and copy assignment operators, as well as usually enforces const correctness. There is some added noise in the code due to the read() and write() calls, but it’s a trade-off I am willing to live with.

Usage is like (edit : code snippet put to pastebin.com because the WordPress code formatting doesn’t like C++ templates…) :

http://pastebin.com/b9sMRBm0

Almost all use of the envelopes in HourGlass was changed to be via the read() and write() methods. This took me a few hours and wasn’t exactly “fun”, but I think the results are worth it. The undo (or version) history for the envelopes is now a lot cleaner, no tricks with byte arrays and zip compression are needed and possibilities for various nice tricks now exist. For example, the state of just a single envelope could be easily and cleanly retrieved back from the history. Memory usage of the history entries has dropped to at least 1/3 of the old system. Adding more state besides the automation envelopes, like effects settings, has become a more plausible option too. I am somewhat on the fence about the effects settings though. I haven’t always found it useful for example in Cockos Reaper that all tweaks in plugins (or the mixer) go into its undo history. However, the way the state versions are now handled in HourGlass, it should be fairly trivial to implement it so that things are always stored, but can be optionally recalled from the history. Unneeded states can also be removed from the history by user action.

The problem with the atomic reference counters is mostly negated by HourGlass having to synchronize access to the objects anyway via mutex locking because I allow changing the states during audio playback. (That is, whatever performance decrease there exists due to thread synchronization, already would exist in any case.) It could be argued that the mutex locking isn’t appropriate to begin with in a realtime audio playback context, but so far I really can’t say I would have seen any really ill effects out of it, barring some obvious programming errors. I have been on the side searching and fixing issues in the code where the mutex locks could be held too long and replaced them with much faster code that has predictable characteristics.

Thanks to Sean Parent for his talks, materials and emails on copy-on-write (and on other stuff). Apologies for any errors, omissions and inconsistencies.

 

Posted in Uncategorized | 1 Comment