Azure function apps: creating a shared .csx file that uses an external library

In this post we are going to discuss how to make multiple C# Azure functions use a shared .csx that uses an external library.

The problem

Imagine the situation is like this, you have two functions in your function app that each take a blob as an input and output a bunch images with different sizes. These functions would have different inputs and outputs, say that take their input from different queues and output the images in some blob storage path.

The resizing library of choice is ImageResizer, which is available on NuGet. To install it, we add a project.json file in the root of our function with content like this.

{
  "frameworks": {
    "net46": {
      "dependencies": {
        "ImageResizer": "4.0.5"
      }
    }
  }
}

This automatically installs and adds a reference to ImageResizer to our function so we can start resizing the images.

So these two functions are essentially the same, they take an input image and output a bunch of resized images, so it's sane that we extract the common piece of code (the resizing part) into its own module.

The catch here is that the common code have to use the ImageResizer library, and the only way to install and manage it automatically is to use the project.json file.

The Solution

Some initial steps are required to make this work, so here are the prerequisites.

First thing is to add the shared file to your function app so that both of your functions can reference it.

• go to https://portal.azure.com/ and login to your account.
• Open Function Apps, choose your function app then click the Platform Features tab from the top navigation bar.
• From the DEVELOPMENT TOOLS section choose Advanced tools (Kudu).
• From the top menu, Debug console -> CMD.
• navigate from the file explorer, choose site -> wwwroot. This is where all your functions live.

Now in the console

> mkdir shared
> cd shared
> touch resize.csx

This creates a file called resize.csx inside the shared folder and that's the file we will be using for our image resizing logic.

Now, the solutions.

One possible solution would be to downlad the .dll file for the library and upload it to the shared folder then import it inside the resize.csx file. You wouldn't then need the project.json file. This would work but you will be losing the advantage of package management that NuGet gives you, so I can't recommend this approach.

A better solution is to take advantage of the fact that the loaded .csx files are much like importing header files in the C language, in which they become part of your file and get access to all available namespaces.

We will start from the docs, which mentions that you import a shared library with the #load directive.

So in the run.csx file of one of the functions, import the shared file. Assuming that it has a function called resize that takes an image as a Stream and an array of sizes to output, it would look like this.

#load "../shared/resize.csx"

public static void Run(Stream someImage) {
    resize(someImage, sizesArray);
}

So this way the shared resize function is available in our function app and the ImageResizer is available to the resize function, due to the fact that they share the same context.

Then for any other functions that need to use the resize function, add the project.json file with the ImageResizer package and you're good to go.

That's it. These are the simplest solutions to deal with that situation. Sure there are other solutions out there, so if you have some solution that you would recommend please share it with us.

React: Parent-child lifecycle methods and refs

Introduction

This post is going to cover the order in which lifecycle methods get called in parent components and their children and where the ref callback fits in this flow, with a little story about passing a ref from a child to a sibling of that child which we are going to solve out gradually.

Backstory

Here's a little context for what got me to write this post, if you are more interested in the meat and potatoes of the thing, you might skip this section, altough things should be much clearer if you read it.

I'll clarify why things behave like this at the end of the post.

Me at work, the design has put us in a situation where a button has to focus a text input that's inside a container that's a sibling of the button's container component. I know that's confusing, so here is an image to illustrate it.

As y'all know in React, focus == ref.

So the parent had to reach into the CatList's text input to get a ref then pass it to the buttonBar.

Approach 1

---

We can go get the starting point straight out of the official docs

class Parent extends React.Component {
  render() {
    return (
      <div>
        <ButtonBar/>
        <CatList getRef={input => this.catInput = input}/>
      </div>
    );
  }
}

class CatList extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      cats: [
        "Cat1"
      ]
    };
  }

  render() {
    return (
      <div>
        <ul>
          {
            this.state.cats.map((cat, i) => 
              <li key={i}>
                <p>{cat}</p>
              </li>
            )
          }
        </ul>
        <input type="text" ref={this.props.getRef} />
        <button> Send </button>
      </div>
    );
  }
}

class ButtonBar extends React.Component {
  render() {
    return (
      <button> Focus </button>
    );
  }
}

So We've sent the getRef function to the CatList component which is going to extract the DOM ref from the input and pass it up to the catInput property of the Parent.

So far so good. If we want to set focus to the input when the parent mounts it's as easy as

// In the Parent component
componentDidMount() {
  this.catInput.focus();
}

That works, the input is focused whenever the parent is mounted.

So we can pass the property down to the ButtonBar like that, right?

class Parent extends React.Component {
  render() {
    return (
      <div>
        <ButtonBar inputRef={this.catInput}/>
        <CatList getRef={input => this.catInput = input}/>
      </div>
    );
  }
}

class ButtonBar extends React.Component {
  constructor(props) {
    super(props);
    this.focusInput = this.focusInput.bind(this);
  }

  focusInput(e) {
    this.props.inputRef.focus();
  }

  render() {
    return (
      <button onClick={this.focusInput}> Focus </button>
    );
  }
}

Wrong, that would give an error that your props.inputRef is undefined.

Approach 2

---

Use the parent component's state to store the ref and pass it down to the other sibling.

class Parent extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      catInput: null
    };
  }

  componentDidMount() {
    this.setState({
      catInput: this.catInput
    });
    this.catInput.focus();
  }

  render() {
    return (
      <div>
        <ButtonBar inputRef={this.state.catInput}/>
        <CatList getRef={input => this.catInput = input}/>
      </div>
    );
  }
}

class CatList extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      cats: [
        "Cat1"
      ]
    };
  }

  render() {
    return (
      <div>
        <ul>
          {
            this.state.cats.map((cat, i) => 
              <li key={i}>
                <p>{cat}</p>
              </li>
            )
          }
        </ul>
        <input type="text" ref={this.props.getRef} />
        <button> Send </button>
      </div>
    );
  }
}

class ButtonBar extends React.Component {
  constructor(props) {
    super(props);
    this.focusInput = this.focusInput.bind(this);
  }

  focusInput(e) {
    this.props.inputRef.focus();
  }

  render() {
    return (
      <button onClick={this.focusInput}> Focus </button>
    );
  }
}

This works, but not for the reason you might think it works for. In fact if we set any state on the parent component, and pass the this.catInput property of the Parent (like we did in approach 1) it will work.

The thing is, setState causes a re-render, so the props are sent correctly (next section explains why the props went right this time).

That means we can replace the code inside componentDidMount of the Parent component with a forceUpdate and it will work.

// Parent component
componentDidMount() {
  this.forceUpdate();
  this.catInput.focus();
}

And in the render method we pass in the class property this.catInput to the ButtonBar.

// Parent component
render() {
  return (
    <div>
      <ButtonBar inputRef={this.catInput}/>
      <CatList getRef={input => this.catInput = input}/>
    </div>
  );
}

Meat and potatoes

To understand the reasons behind what worked and what didn't work in the previous story we can first start with how lifecycle methods are called. The order is as follows:

• The render method of the parent (passing the props).
• The render method of the children.
• Ref callbacks in children.
• The componentDidMount method of the children.
• The componentDidMount method of the parent.

So to wrap up

• the props are already sent before any of the hierarchy's componentDidMount methods are called.
• The ref is not available until just before the componentDidMount of the children is called.
• componentDidMount of the Parent is then called after all its children had called theirs.

That's why we could read the input ref in the componentDidMount of the Parent but the input prop was undefined when passed to the other component (the ButtonBar's props.inputRef).

Causing a re-render makes the prop be sent with the right value, because re-renders always give the child components the up-to-date values of their props, and that's what forceUpdate did in the last example above.

forceUpdate might not be the best solution to this kind of situation, I'd like to know what your approach would be for that matter.

That's it! I hope you enjoyed the little story.

if you have any suggestions, want to point out mistakes or share knowledge, you can always leave a comment.

Javascript - Two properties to keep in mind when using block-scoped variables

let and const have some interesting properties other than their operating scope, this post discusses two of these that might save you some head-scratching.

They don't add properties to the global object

Unlike var, even in the global scope, block-scoped variables do not attach properties to the global object.

This might lead to something like this.

Code snippets in this post use node as their execution environment, replace "global" with "window" for the browser.

const c = "constantC";

console.log(global.c); // undefined

global.c = "globalC";

console.log(c); // Prints "constantC"
console.log(global.c); // Prints "globalC"

So, as you can see on line 7, whenever you refer to the variable name, Javascript gives you the const value. If you want the global property you reference it from the global object.

This can lead to the following weird scenario

v = "globalV";

console.log(v); // Prints "globalV"

const v = "constV"; // No problems at all

console.log(v); // Prints "constV"

console.log(global.v); // Still "globalV"

You can indeed, in the global scope, declare a const/let variable with the same name as a global variable (memory leak warning here).

As you know, var behavior is totally different in these scenarios.

A use case

imagine this..

You're stuck with a project with some messed up dependency injection, so one module assumes the other module is in the global scope and uses it directly (talking about old school IIFE mostly)..

When wanting to test that module in isolation is it often a pain to do so, you want to mock/ stub/ fake/ double/ whatever that module.

using the above property can make you set a tempoary global variable for use by the first module, then comes the real one and declare itself as a const in the global scope with no problems.

This, however will not help you re-order tests or run them in parallel, so it's always best to architect the code in the right way (obvious, but not always as easy as it sounds).

They have a temporal "dead zone"

Block-scoped variables will not hoist (In the way we're used to, anyway) to the top of the block, causing a temporal dead zone between the top of the block and variable initialization.

when a Var is defined in code, it is automatically initializaed with undefined, block-scoped variables do not follow the same behavior, which means that declaration and initialization phases are decoupled, which in turn causes an error thrown when trying to access the variable anywhere above its initialization line.

console.log(letV); // Throws an error: letV is not defined

let letV = 2;

A use case

Imagine you want to do something if some object is available to use, like if exports is present in the global scope, then attach members to it.

var exports = exports || null;
if (exports) {
  exports.thing = myThing;
}

This is something you can't do with a const or let without throwing an error.

That's it.

Hope that helped. Please share your thoughts in the comments.

WebAssembly - sending a Javascript array to wasm for modification and getting back the modified version.

Alright so WebAssembly has hit the MVP and is enabled by default in more than one browser right now, so starting to use it makes sense.

One of the main functions of WebAssembly is to let it handle the heavy lifting part of the code.

This post is going to take you through sending an array to a 'wasm' function, make some crazy low level stuff on its elements and finally reflecting that change on the web page with Javascript.

Note that this post explains how to send an array that's already filled, which involves copying the elements into the WebAssembly memory space. In a lot of cases, though, you would want to skip the copying part and just build the array incrementally directly into the WebAssembly memory space (see the JS code near the end).

For the sake of simplicity, his post assumes that you are using C/C++ with Emscripten to generate the helper JS file.

C

Let's start at the C level (sorry)

#include <stdint.h>

int addThree(uint8_t *buf, int len) {
  uint8_t *item;
  uint8_t *end = buf + len;

  for (item = buf; item<end; item++) {
    *item += 3;
  }

  return 0;
}

Let's break things down..

On line 1, we include the stdint library which gives consistent operation across different machines.

On line 3, we define our function, it takes two parameters:

• a pointer to the start address of the array in memory (which is technically the location of index 0). Array pointers have to be one byte in size.
• and the number of elements in this array (to help us loop through the items).

We then define two pointers to the start and end location of the array in memory (it is safe to add together diffreent integer sizes).

We then loop through the array items and do the crazy stuff (like adding 3 to each element in this case) and just return 0 to say that we are done.

Look out for overflows, like in the above example, we have used the unsigned int8 integer to hold the data, which can store integers from 0 to 255, so adding a 3 to say a 253 will cause an overflow and get you into trouble.

Alright, One thing left, we've got to compile that code and export the addThree function.

assuming that you already have emcc into your PATH, compile the C file with this (I choose the name "freeThree.c", because it rhymes).

 emcc -o three.html freeThree.c -s WASM=1 -s EXPORTED_FUNCTIONS="['_addThree']"

Here we are just telling the compiler to use native WebAssembly (and not asm.js) and we tell it to expose the addThree function.

Notice the "_" before the function name, this is very important.

While we are here and before we go to the Javascript side of things, if you may have to pass large arrays to this function, consider making room for the extra memory.

one technique that I use is to use the ALLOW_MEMORY_GROWTH flag, run the web page, and keep an eye on the console, it's going to give you the a number every time is grows the memory. Note that when your app demands extra memory, the size of memory doubles rather than keping up with the exact demanded size, still you can get a nice approximation about what your maximum memory size should be. Add this to the end of the previous compile command after EXPORTED_FUNCTIONS="['_addThree']".

-s ALLOW_MEMORY_GROWTH=1

ALLOWMEMORYGROWTH prevents some optimizations that the compiler can make, so after you have decided that your app will use a maximum of X memory it's better to use that number for specifying the static memory of your app. So instead of the ALLOW_MEMORY_GROWTH flag, use this

-s TOTAL_MEMORY=X

Javascript

Now let's shift our focus to the Javascript level.

After you've run the above command, emscripten should have outputted a nice bloated three.html file with a .js file with the same name (along with the .wasm file itself), this .js file is gonna make our life a lot easier when dealing with the weird WebAssembly memory model.

We won't change anything in the script file, it's just our interface to the WebAssembly binary (.wasm file), it fetches, compiles and instantiate the wasm file and gives us a nice and clean Module object ready to be used.

Now open the three.html file and inside the first <script> tag, right before the closing </script> tag which is right above the other <script> tag (I hope this has clarified more than confused..), create a function with the following Javascript (you can use it as an event handler, or just shove it into the Mosule's postRun array, which is on around line 1226 in your three.html file, to run it as soon as the Module loads)

const myArray = new Uint8Array([100, 200, 250]);

const buffer = Module._malloc(myArray.length);
Module.HEAPU8.set(myArray, buffer);

Module.ccall('addThree', 'number', ['number', 'number'], [buffer, myArray.length]);

console.log(Module.HEAPU8.subarray(buffer, buffer+myArray.length));

Module._free(buffer);

Here's what's going on..

On line 1, we create our array that we wish to send to the C function for manipulation.

Please note that AFAIK (and according to the emscripten docs), you can only send Uint8Array or or Int8Array. If you have figured out a way to send other typed arrays please share it with us.

Then on line 3, we allocate enough memory (in bytes, it's just the length of the array here because each element is one byte in size) to hold our array elements.

We then actually copy the elements from the array into the buffer we have just created, which means that we are copying the data into the WebAssembly Module's memory space. While the module's memory is represented as a single ArrayBuffer, Module.HEAPU8 is the Uint8 view of that buffer, so it's just a normal Javascript typed array built around that buffer, so in turn, it has the set function.

After that we use the utility function ccall to call the C function, passing in the buffer to act as a pointer to the first element in the array.

ccall docs and other options of calling the C function can be found here.

Line 8 uses the subarray function to peek at the array items while they are sitting happily in their ArrayBuffer, you can use subarray or slice to deal with the modified elements. You can find more details here, but the bottom line is if you just wanna look at the elements or loop over them or the likes and then throw them, use subarray, it's way faster then slice.

Finally we must free up the allocated memory space (a la C malloc/free functions).

The catch

Copying the data (with set) takes time, if you have a huge array this can easily be your app's performance bottleneck.

Imagine you have a canvas that you want to manipulate which is 4000 in width and 3000 in height, this canvas has 4000*3000 pixels, which is 12 million pixels. Now you want to get the ImageData array to be processed by the fast C function, this array gives every pixel 4 entries, which is 4 bytes, now we have 48 million entries/bytes that we have to copy into the memory space of WebAssembly. On a modern machine this takes something close to 2 seconds!

So three's a tradeoff happening here, just be sure that copying the data isn't going to be your bottleneck (like, don't use it for adding 3 to the entries). If so, you better off doing it with Javascript, it would be a lot faster (You can also use workers to get some concurrency).

Hope that helps clear things up. please share your suggestions and questions below.