What Do I Need To Know About JavaScript Part I

This is a frequent question I receive from those wishing to parlay a knowledge of Javascript into a long career as a developer. Speaking of long careers, mine currently spans 35 years of professional experience.  I began this long journey writing assembly-language math libraries for supercomputers.  Of course, how you start is rarely indicative of how you will end.  Since then, I’ve also worked with Fortran, C, C++, Java, JavaScript (and Mocha/LiveScript), ActionScript (1/2/3), and Typescript.  This reminds me of the line from RUSH’s popular Tom Sawyer song, “changes aren’t permanent, but change is.”

Whenever I take on the task of learning a new language, I’m always interested in the internals of what makes that language work.  What was the design philosophy of the language? Is there a single concept or a group of core concepts that explain how and why the language/environment produces run-time results?  I’ll discuss three areas in this post and then provide a second post on additional topics if there is sufficient interest.

Functions are first-class language objects in Javascript and almost everything of importance in an application revolves around them.  So, I’ve personally found understanding function execution context to be highly valuable in constructing an understanding of Javascript internals.

EXECUTION CONTEXT

Now, I don’t like re-inventing the wheel, so wherever possible, I will provide links to articles on individual topics that I’ve found to be not only accurate, but very helpful in leading to an in-depth understanding of that topic.  And, when it comes to execution context, I still think Dave Shariff’s article is one of the best,

http://davidshariff.com/blog/what-is-the-execution-context-in-javascript/

This article is well worth spending a lot of time understanding, particularly the section on hoisting.

Following are a couple key takeaways from that article.  You will hear the terms ‘context’ and ‘scope’ mentioned a lot and these concepts are not unique to JavaScript.  A ‘scope’ is simply a well-defined area in a run-time environment that is used to look up variable references.  Let’s not overcomplicate things.  If I say to someone, “Hey, look at that red Mustang,” but there is a red Mustang in both the Kroger and Walmart parking lots, then I need to specify which lot (or scope) in order to make the vehicle (variable reference) unique.  I may have to be even more specific and identify the red Mustang at the south end of the Kroger parking lot.

There are three possible contexts in Javascript,

– Global (the application has to start somewhere)

– Function (program flow enters a function body)

– Eval (execution inside an internal eval function)

So, what is ‘scope’ in Javascript?  At any single point in the flow of program execution, it will be one of these three contexts (parking lots).

Execution Context consists of three items,

  a) A scope chain, represented as an ordered list of variable objects (V0), or variable tables, that are ordered from the function’s direct parent back up to the global context.

  b) An internal variable table or VO for the function itself that includes the arguments Array and an entry for each variable defined internally in the function.

  c) A unique resolution to the self-referential pointer (i.e. ‘this’).

The final item requires some additional explanation.  We must be able to uniquely and predictably resolve references such as this.x inside a function, so how Javascript assigns the self-referential pointer is very important.  It is also unique to JavaScript in that the assignment depends on how the function is invoked.  I’ve found this knowledge to be rather important as it can explain a lot when you are forced to debug/modify code written in the past by one or more devs.

Here are the ways JavaScript assigns the self-referential pointer based on function invocation:

1 – Method invocation.  The function is a property of an Object.  ‘this’ is bound to Object in which the function is a property.

2 – Function invocation. The function is defined outside an Object and invoked by direct call.  ‘this’ is bound to the global (typically Window) object.  Crockford correctly pointed this out as a flaw in the language and one that can result in undesired (and not easily understood) results.  Let’s look at some examples.

Copy and paste the following code block into your browser console and execute,

var x = 2;
function foo() {
  var x = 1;
  console.log('x in foo:', this.x);
};

foo();

The function, foo, is defined ‘standalone.’  When it is called on the last line, that is known as Function invocation.  The execution context for the foo function consists of a scope chain containing only the global context.  The VO consists of a single variable, x, which is not defined until the activation stage of execution context.  The arguments Array is empty.  The self-referential pointer is set to the global context.

So, even though there is a reference to a variable named x inside the foo function, resolution of this.x comes from the global context.  So, ‘x’ is logged as 2 in this example.

Suppose the self-referential pointer was removed and the example was,

var x = 2;
function foo() {
  var x = 1;
  console.log('x in foo:', x);
};

foo();

Based on what we have studied from the concept of execution context, x is defined inside foo’s VO as 1 during the activation stage of execution context.  With only a ‘raw’ variable reference, the first attempt at name binding or variable lookup is from the function’s VO, from which a variable with that name exists.  In this case, x is logged as 1.

Now, let’s change the example to

var x = 2;
function foo() {
  console.log('x in foo:', x);
};

foo();

In this case, the first lookup for x inside foo’s VO fails.  Subsequent lookups proceed along the scope chain defined during the activation stage of execution context.  The first (and only) VO in this chain is the global context, in which the variable, x, exists.  In this instance, x logs as 2.

And, since you understand that unique assignment of the self-referential pointer is part of a function’s execution context, and you understand the steps in variable lookup, you should understand that in this example,

var y = 2;
function foo() {
  var x = 1;
  console.log('x in foo:', this.x);
};

foo();

The variable, x, now logs as undefined as there is no such variable in the global context.

3 – Constructor invocation.  The function is defined and stored in a variable and then invoked with the new keyword, i.e.

var x = 7;
var y = 8;

var Foo = function() {
  this.x = 1;
  this.y = 2;
};

Foo.prototype.z = function() {
  return this.x + this.y;
};

var bar = new Foo();
console.log('z: ', bar.z());

This is an important example as it is the means by which traditional classes are approximated in JavaScript, which is built on prototypal inheritance, i.e. Objects simply inherit from other Objects.  When constructor invocation is applied, an addition is made to the function’s VO that is a reference to the function’s prototype.  The self-referential pointer is then set to that reference.

So, in the above example, the variable z logs as 3.  However, in this example,

var x = 7;
var y = 8;

var Foo = function() {
  var x = 1;
  var y = 2;
};

Foo.prototype.z = function() {
  return this.x + this.y;
};

var bar = new Foo();

console.log('z: ', bar.z());

z logs as NaN.  Why?  The function’s VO includes table entries for variables x and y, however, this operation does not call a method created by the function, but one that exists on its prototype.  Variables x and y are never defined on the prototype, so the addition can not be coerced to a meaningful result (more on that later).

4 –  Apply/bind – Apply/Call/Bind allow a function to receive a specific context for the self-referential pointer.  This is the easiest invocation example to understand.

5 – Lexical propagation, aka arrow functions.  Lexical propagation is simply a fancy name given to passing of the parent context of a function to the function itself.  It’s simply a replacement for assigning a reference to the ‘this’ pointer to a variable, i.e. var parent = this or var that = this and then using the parent variable inside a child function body.  Instead of this.x inside the function body, the code references parent.x, for example.

Lexical propagation via arrow functions allows a function to be defined independently using references such as this.x .  The function may be imported into a Typescript class, for example, and lexical propagation provides an automatic method closure.

This statement leads us right into the next section 🙂

SCOPE AND CLOSURE

Well, now, we get into some of that computer-science stuff, but some principles are best understood from a more  fundamental, language-independent context.  Some of what you may read on these topics in Javascript are wrong or only partially correct.

Name-binding or the mapping of a name such as a variable to a specific value is important in any language. Scope is a specific region of the program in which a specific name-binding is applicable. Scope is normally identified at the syntax level, although it is applied at run-time.

Shariff’s article was written before ES6 and at that point in time, variables in Javascript were scoped at the function level.  A variable x, for example, might be defined inside an if-block, but a property for that single variable is created inside the function’s VO during the activation stage of execution context.  That same variable might be assigned a new value outside the if-block, but there is still only a single entry in the VO that serves to bind the name, x, to a single value inside the function, regardless of where the variable is declared inside the function.

ES6 introduced let and const variables that are block-scoped, which makes the function’s VO table more complicated.  Name binding during the process of function execution can now depend on a specific, basic block inside the function body.  This is illustrated by example,

var x = 2;
function foo() {
  var x = 1;
  var y = 2;
  let z;

  if (y > 0) {
    let x = 3;
    z     = x + 2;
    console.log('x/z inside if:', x, z);
  }

  console.log('x in foo:', x);
};

foo();

There are two references to the variable, x, inside the function, foo.  So, how is name-binding applied?  There are no references to the ‘this’ pointer, so the variable is looked up first inside the function’s VO table.  However, there are two scopes in which x can be resolved.  The first is block-level and is inside the basic block encapsulated by the if statement.  Name-binding begins at the current scope, which is the if-block when the first console log is encountered.  Since there is an immediate resolution of the variable reference, the value of 3 is logged for x.  However, that scope only persists for the duration of the if-block.  If the let x = 3 statement were removed (try this) then name-binding is applied at the next-highest scope, which is the function level.  There is a match at that level, so x logs as 1.  If the var x = 1 statement were removed, there is no match anywhere in the function’s VO, so lookup proceeds along the scope chain created during the activation phase.  As with the other examples, there is only one VO in that chain, which is the the global context.  There is a match on x in that VO and the value of 2 is applied to the variable, x.

So, in this example, we could say that there are three scopes or regions of the program in which name-binding can be applied, each of which produce a possibly different resolution for the same variable name, or the variable may not be bound at all. These scopes are the if-block inside the function foo, the function itself, and the global context.  Think of it as dividing the parking lot into north and south partitions and then further specifying an east/west partition for the south partition.  These union of these partitions may or may not be the entire parking lot, so we may think of the entire parking lot as the global context.  The variable ‘red Mustang’ is uniquely resolved (or not resolved at all) based on a specification of scope.

Now, what about those pesky closures you may have heard of or read about somewhere else?  Yes, here we go with more computer science stuff …

I like the Wikipedia definition of closure the best, which is simply an implementation of lexically scoped name binding in languages with first-class functions.  Lexical scoping refers to independence from a run-time environment and dependence upon the lexical or program-level structure of the language to define scope.  A closure is applied to a function as an independent entity by creating a combination of the function with a context for name-binding.  From the prior material, we can see that a function closure is created by the totality of the environment created during execution context.

When closures are applied to a method of a ‘class’ (recall this method exists on the function’s prototype), this combination of function and context is called a method closure.  ActionScript, for example, was ECMAScript compliant since version 5 of the Flash player.  ActionScript 2 introduced the syntactic sugar of classes that compiled down to AS 1 in a similar manner to how TypeScript compiles down to ES5.  Classes are not first-class language objects in any current version of JavaScript and this was also true of AS 2.  To use a class method as an event handler, for example, we created method closures using a helper function called Delegate.

For the remainder of this article, we can infer that closure always means function closure.  Closures may be created and manipulated by developers, generally for the purpose of variable privacy or to avoid undesirable side-effects of code that may seem to perform a required task, but do not execute as desired.  Closures may also be used in the technique of partial application of functions and the following is a good introduction to the topic,

https://medium.com/javascript-scene/master-the-javascript-interview-what-is-a-closure-b2f0d2152b36

While you can understand a lot (theoretically) about closures through function execution context, there are some subtle issues with how they are created and applied that are best illustrated by example.  Consider the construction-invocation example, above, with an expanded console log.

var x = 7;
var y = 8;

var Foo = function() {
  this.x = 1;
  this.y = 2;
};

Foo.prototype.z = function() {
  return this.x + this.y;
};

var bar = new Foo();
console.log('z: ', bar.z(), bar.x, bar.y);

Execute this inside your browser console and note that z, x, and y, log to 3, 1, and 2, respectively.  While it may be tempting to consider the variables x and y to be ‘closed’ inside Foo, they are completely visible (and can be manipulated) outside that function,

var x = 7;
var y = 8;

var Foo = function() {
  this.x = 1;
  this.y = 2;
};

Foo.prototype.z = function() {
  return this.x + this.y;
};

var bar = new Foo();
bar.x   = 10;
bar.y   = 11;

console.log('z: ', bar.z());

Ouch.  Now, z is 21.

While the latest version of JavaScript supports truly private variables in ‘classes’ by prefixing with # symbol, you may be called upon to work with code created by other developers, perhaps many years ago, in which manual closures were created to facilitate data privacy.  For example,

const point = function(x, y) {
  let xCoord = x;
  let yCoord = y;

  return {
    getX: () => xCoord,
    getY: () => yCoord
  };

};

const somePoint = point(0, 3);
const xValue    = somePoint.getX();
const yValue    = somePoint.getY();

console.log('values:', xValue, yValue, somePoint.xCoord, somePoint.yCoord);

Note that this logs 0, 3, undefined, undefined.

The accessor functions, getX() and getY() reference variables defined in the VO of the point function, although we can not directly access those variables from outside the point function.  This is unlike the prior example, which utilized the Foo function’s prototype (which any Object may access) to create variables and methods.  Sometimes, functions such as getX() and getY() are referred to as privileged methods.  These methods were defined at a level in which name-binding of the variables xCoord and yCoord could be resolved to known, defined values.  So, they are ‘privileged’ to have access to that information during their own execution context activation stages as a result of being defined inside the function, point.

This type of closure can also be used to create stateful functions, sometimes in tandem with an IIFE (Immediately Invoked Function Expression).  An example I use when teaching this topic is a Wythoff iterator.  Strictly speaking, this iterator does not produce sequences that are exact rows of a Wythoff array, but it is Fibonacci-like and accepts two starting values, just like the Fibonacci sequence.  Subsequent values of the iterator are obtained by adding the two previous values.  This example, with starting values of 0 and 1, could be considered a Fibonacci iterator.  Now, the Fibonacci sequence is a recurrence relation with constant coefficients and all such relations have a closed-form solution (see Knuth TAOCP).  The closed-form solution for the Fibonacci sequence is Binet’s formula, but that is not used in this example.  Instead, we maintain internal variables, first and second, which are overwritten each time the iterator’s next() function is called.  This code is crude, but serves to illustrate the desired concept,

const iterator = (function (f, s){
  let first  = f;
  let second = s;

  return {
    next: () => {
      const tmp = first + second;
      first     = second;
      second    = tmp;

      return tmp;
    }
  }
})(0, 1);

console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next(), iterator.first, iterator.second);

The IFFE results in immediate invocation of the function with the arguments 0 and 1.  This triggers the activation and execution phases of that function’s execution context in one step, so to speak.  Space for the function variables first and second is created in the VO table during the activation phase.  Specific values of 0 and 1 are assigned to those variables in the execution phase.  Since the result is stored in a variable, iterator, we might think of this process as defining a state for the function that influences its execution on each call of the privileged method, next().

Run this code in a fresh browser console and you should see the result,

next: 1
next: 2
next: 3
next: 5
next: 8
next: 13
next: 21
next: 34
next: 55 undefined undefined

The final line once again illustrates that we can not directly access the internal variables, first and second.  State is maintained between function calls.

This example can be extended more complex examples and many have labeled this general approach the ‘Revealing Module Pattern,’

https://weblogs.asp.net/dwahlin/techniques-strategies-and-patterns-for-structuring-javascript-code-revealing-module-pattern

Loop induction variables that serve as function arguments provide another interesting example of closures and a segue into some of the newer language features.  Consider a classic example,

var funcRefs = [];
for (var i = 0; i < 5; i++) {
  funcRefs[i] = function() {
    console.log("value is: " + i);
  };
}

for (var j = 0; j < 5; j++) {
  funcRefs[j]();
}

Run this in a browser console and you will see ‘value is: 5’ repeated five times.  The variable, i, is scoped to the global context in this example, or a level just above the first for-loop.  That variable is maintained after loop execution is terminated and its value is 5.  Why 5?  The induction variable is incremented before the test for loop termination is made.

When each anonymous function is created, its scope chain contains the global context as part of the execution context created during the activation phase.  When each function is executed, variable lookup starts at the anonymous function’s VO.  There is no reference to a variable, i, so lookup proceeds up the scope chain. That name is bound in the global context to the fixed value, 5.  So, in the second loop, each function resolved the variable, i, to the same value.

I’ve seen a number of resolutions to this problem, even to the extent of IIFE’s created inside the loop.  I’ll show you the ‘classic’ or ‘old-school’ resolution to the problem and then discuss a couple more modern solutions, should you encounter this case in code you are assigned to debug and/or modify.

From the example above, we can use the internal VO table of a function as a means to store ‘state.’  The value of the induction variable for the first loop can be passed to a function factory and then stored in the VO for that function,

var funcRefs = [];
function fcnFactory(i) {
  return function() {
    console.log("value is: " + i);
  };
}

for (var i = 0; i < 5; i++) {
  funcRefs[i] = fcnFactory(i);
}

console.log('variable i is:', i);

for (var j = 0; j < 5; j++) {
  funcRefs[j]();
}

Verify that this displays the result that was originally intended,

variable i is: 5
value is: 0
value is: 1
value is: 2
value is: 3
value is: 4

The problem in this example is function-level scoping.  If we could make the first loop induction variable block-scoped (as in ES6), then the example should work as desired.  This is due to the fact that when the anonymous function is created, the ‘outer’ scope is the context containing the value of the induction variable at each loop iteration.

var funcRefs = [];
for (let i = 0; i < 5; i++) {
  funcRefs[i] = function() {
    console.log("value is: " + i);
  };
}

console.log('variable i is:', i);

for (var j = 0; j < 5; j++) {
  funcRefs[j]();
}

Note that in this example, the loop induction variable does not exist outside the loop; it can only be bound inside the loop and each iteration produces a different bound value.

We can also avoid problems of this nature with automatic closures provided by Array methods such as forEach.  Each invocation of the callback function is given its own context (or closure) based on the variable(s) used in the function argument(s). 

COERCIONS

This is my final must-know topic in part I of this series, especially since JavaScript likes to ‘make things work,’ even things that on the surface seem to make no sense.  In order to better understand what I’m talking about, let’s discuss the difference between casts and coercions.

A cast in any language is a ‘hard’ statement that is intended to force the compiler to generate code to convert a variable of one type into a variable of another type.  In C++, for example, a developer may force a variable to be converted from a double to an int by a statement of the form,

n = (int) z

where the variable, n, is typed int and the variable, z is typed double.  This statement could be interpreted as “yes, I know z is a double-precision float, but FORCE it to be an integer and then assign the result to the variable, n“.  Casts, while specific, still require some understanding of how the cast is performed.  For the above example, does the compiler truncate or round the floating-point number?

At least a cast is a specific statement by the developer that indicates intent.  A coercion on the other hand is performed silently by the environment and some developers may not even know that the coercion occurred.  Coercions occur when assignments are made from one variable type to another, during some operations, and even during comparisons.  The latter case often takes new JS developers by surprise and is the primary reason that strict equality (triple-equals) tests are preferred over basic equality tests (double-equals).

Here is a great example.  Study the table in this link and see what can happen when variables of different type are compared with a double-equals test.

https://dorey.github.io/JavaScript-Equality-Table/

Coercions occur ‘under the hood,’ sometimes without a developer understanding that they are happening at all.  Internally, JavaScript ties to convert one or more variables of an existing type into a type that somehow makes the operation or assignment ‘work,’ even if the definition of ‘work’ does not make sense to us.  There are also some instances in the language that are somewhat in-between a cast and a pure coercion.

One example is the infamous double negation, ie. var x = !!y.  There is no double-negate operator and I personally do not like this practice.  But, some people like writing tricky or excessively compact code, so you may encounter this syntax in pre-existing code.  The first negation attempts to convert the existing variable, y, to a Boolean primitive and then logically negates the result.  Something that is truthy is converted to Boolean false and something falsey is converted to Boolean true.

The second value logically negates the first, so the final result of the coercion is that truthy variables are converted to Boolean true and falsey variables are converted to Boolean false.

Another example is placing a plus or minus sign before a string variable, i.e. +x or -y (the unary prefix operator).  This operator attempts to coerce the string into a positive or negative number.

JavaScript actually has a very organized system for converting complex objects to primitives and then converting from one primitive to another.  I remember an discussion with a somewhat new JS developer (less than five years experience) who was frustrated in an interview when asked to provide the result of the operation [] + {}.

I think the question, as posed, is a bit ridiculous as it infers a sense that someone might intentionally attempt such an operation.  It could, however, occur as a side-effect in a legacy system were variables were generalized from primitives such as string or number into Object and/or Array. These variables could be passed in some rare outlier case to a function that expected primitives, but received something more complex.  JavaScript is in the business of results.  Some result.  Any result.  Making sense does not matter 🙂

So, it does help to understand the internals of this coercion system, especially if you are called upon to work on code that has existed for a long time and has been revised by a number of developers over the years.  Now, before we being, memorize the primitive types in Javascript, namely undefined, null, boolean, number, string, bigint, and symbol.  Everything else is an Object (including Arrays and functions).  Primitives are immutable and can not be changed, and we’ll discuss that more in part II. 

On the subject of coercion, primitives do not have properties, so how is it that we can access the length property of a string?  JavaScript will happily and temporarily coerce a primitive to an Object, imbue the temp Object with properties, and then make the Object available for garbage collection after the property access.  Be very careful with the last sentence as it possible to assign properties to a primitive, but access to those properties is undefined.  For example,

var someString = "Math is cool";
someString.numWords = 3;
var words = someString.numWords;
console.log(words);

The variable, words, is undefined.  Try it in your console.  Behind the scenes, JavaScript creates a temporary Object that allows assignment of the numWords property, but no permanent reference is created for that Object.  It is immediately marked for garbage collection.  JavaScript creates a second, new temporary Object in order to allow access of the numWords property, which of course, does not exist.

Now, how does Javascript perform coercions from Objects to primitives and primitive to primitive?  In particular, how does JavaScript perform seemingly crazy operations such as [] + {}?

I’ve found this article to be an excellent introduction to the topic,

https://2ality.com/2012/01/object-plus-object.html

The key takeaways from this article are first understanding important built-in Object methods such as valueOf and toString.  Then, the coercion process can be broken down to convert to primitive followed by convert to similar primitives.  The answer to the above question is covered in that article, namely the string, ‘[object Object]’.

Coercions are subtle because there is no direct indication in the code (such as a hard cast) as to their application.  Even casts may not produce a desired result simply because at run-time, JavaScript attempts to perform the cast and then executes a coercion before completing execution of a specific line of code.  There is only one way to become effective a dealing with coercions and that is to study JS internals and understand where coercions are applied.  That will make it easier to diagnose issues in legacy code and, hopefully, force you to take more time in being explicit about intent when writing your own code.

Well, that brings us to the end of part I and we have not discussed important topics such as event loop, pass by value or reference, data structures such as Set/Map, Promises, and more.  If there is interest, I will author parts II and III of this series an we can dive into those topics along with curated links to quality content that should be part of your regular study.

In the mean time, here’s a preview of what is coming next.  Try predicting and then explaining the output of the following code block.  I regularly use this example in lectures to JS students.

var arr = [1, 2, 3];
var value = 1.0;

var somefunction = function(v, a)
{
  v = 2.0;
  a = [4, 5, 6];
};

somefunction(value, arr);

console.log( "v/a: ", value, arr);

var anotherfunction = function(v, a)
{
  v    = 3.0;
  a[0] = 4;
  a[1] = 5;
  a[2] = 6;
};

anotherfunction(value, arr);

console.log( "v/a: ", value, arr);

Hint: Primitives are immutable and are passed by value. JavaScript creates a separate handle or reference to Objects and passes that reference by value. I like to use the analogy of a pointer variable in C and C++ . So, there is a difference when a pointer’s value changes and thus points to something new vs. dereferencing that pointer. The copy still points back to the original entity, so the dereference affects the original’s value. Okay, that was more like an explanation than a hint, but I hope it stokes your appetite for new content.

Thanks for reading such a long post and best of luck to you in your JavaScript career!

New Articles on LinkedIn

I’ve moved several blocks of content from the algorithmist.net domain into my public profile on LinkedIn.  Here are the articles, which cover a variety of topics including angular, typescript, math, and datastructures.

https://www.linkedin.com/pulse/angular-champion-jim-armstrong

https://www.linkedin.com/pulse/typescript-decision-tree-jim-armstrong

https://www.linkedin.com/pulse/typescript-sequencing-engine-jim-armstrong

https://www.linkedin.com/pulse/reactive-data-driven-finite-state-machine-jim-armstrong

https://www.linkedin.com/pulse/5-sided-die-problem-jim-armstrong/

https://www.linkedin.com/pulse/minimizing-worst-case-complexity-jim-armstrong/

Rotate A Point Collection About Another Point

Well, this will probably be the last, or nearly last post of a busy year that involved a lot of work on gigs and little focus on this blog.  Sorry, maybe next year will be better 🙂

In the previous example, I showed how to rotate a box (rectangle) around an arbitrary point, but the algorithm and code never presumed anything about the geometric nature of the object.  It’s possible to extend the exact same algorithm to a collection of points and that is the subject of the current post.

Instead of maintaining variables for the four vertices of a rectangle, the code was modified to work with an arbitrary point collection.  A RotatablePoint class was used to hold data points and offload some of the computations.  This greatly simplifies the actual demo and provides you with ample means for experimentation.

The online example starts with a small collection of points as shown below.

rotate

These points are rotated about the fixed point, drawn as a red dot.  The drawing may be cleared after each rotation increment or continually updated from the prior drawing as shown below.

rotate1

View online demo.

View source.

Have a Merry Christmas and I’ll try to post more next year!

Geometric Shapes in Freehand Drawing Library

Well, here’s another example that I would not have though appropriate for the Freehand Drawing Library.  Regular  geometric shapes might be drawable as strokes (i.e. press-move-release) with some sort of external control to force straight or constant-angle line segments.  That concept, however, does not seem to fit in a freehand or freeform drawing library.

After some thought, many geometric shapes can be defined (even if somewhat arbitrarily) by a bounding rectangle. The current TwoPoint stroke class could easily be extended to define such a bounding rectangle.  By injecting different drawing engines, it is possible to draw a nearly unlimited number of shapes inside that bounding rectangle.

To illustrate the idea, a GeometricShape class was created that extends TwoPoint.  I developed three sample drawing engines for this stroke, Ellipse, Triangle, and BasicArrow.  Each engine draws the bounding rectangle while the mouse is down and clears it on release.  Pre-draw parameters may be assigned to each engine to further control the shape (i.e. select an isosceles or equilateral triangle or alter arrow shape).

While these engines may seem trivial, they open the door to creating a wide variety of such drawings in the FDL, and, all line decorators are available for each shape.  So, if you want a dashed ellipse or dotted arrow, it’s simply a matter of injecting the desired line decorator.

An astute reader (aka existing beta users) may ask, “what about editing?”  It’s easy to modify or extend these shape drawing engines to return the vertex information for the shape.  This information can be edited and then stored in a value object.  That VO may be sent as auxData in the stroke dataProvider.  If present, the stroke could simply redraw the shape directly from the edited vertex information instead of a bounding rectangle.  It’s all supported in the FDL architecture.

An update is going out to all beta users this morning.

Editing A Cubic Bezier Spline

A couple months ago, this was something I believed I would never say about the Freehand Drawing Library.  Now, the library is and always will be a drawing library. Editing is not part of the core library architecture, however, it is something that requires a level of support inside the library.  In the past, this was accomplished by caching the sequence of mouse motions used to define a stroke.  The points could be edited and manually assigned to a stroke, either all at once or in an ENTER_FRAME event to animate the stroke.

This works cleanly with a simple interface for typical strokes, i.e. touch-move-release motions.  Now that the FDL supports splines and possibly other constructs that stretch the definition of a stroke, what about editing more complex drawings?  I want to maintain a light interface and not make editing operations an integral part of the library.  That inevitably leads to interface and code bloat in an attempt to satisfy every possible combination of customer-created editor.

I think it was the movie ‘War Games’ that popularized the phrase, ‘always leave yourself a back door.’  The back door to stroke manipulation outside normal FDL methods is to use arbitrary name-value parameters.  Every stroke has the ability to access or mutate arbitrary data objects.  It’s only two methods in the API, but they provide a wide variety of capability to custom strokes.

I added a simple spline editor to the PolyLine demo as an illustration.  After creating a spline (by clicking the end button), the knot/tangent display is overlaid on top of the stroke as shown below.

The spline knots are already available since they were manually assigned to the stroke.  The tangent information is entirely encapsulated inside the drawing engine, inside the stroke.  That information is obtained by the demo via a parameter request,

__stroke.getParam( “tangent” + i.toString() );

It is the responsibility of each stroke class to document all custom parameter queries and settings.  The above query returns in- and out-tangent values in an Object and that information is passed to the editor.  Knots may be dragged, which cause a parallel shift in the tangent.  The new knot and tangent information is conveyed back to the stroke with a sequence of parameter settings,

__stroke.params = { “changeKnot”:{vertex:index, x:editor.knotX, y:editor.knotY} };

or

__stroke.params = { “changeInTangent”:{vertex:index, x:editor.inTangentX, y:editor.inTangentY} };

__stroke.params = { “changeOutTangent”:{vertex:index, x:editor.outTangentX, y:editor.outTangentY} };

If a spline drawing engine is assigned to the PolyLine, it passes this information onto the internal FDLIntepolatingSpline instance.  A non-spline engine (line segments) ignores the parameters.

The screenshot below shows the result of a knot drag.

After dragging the middle vertex, the following screenshot shows the result of editing the first-vertex out-tangent.

This is all supported by the existing architecture, but there is one wrinkle.  If the edited spline is to be saved and then redrawn at a future time, we must have the facility to record the tangent edits *and* bypass the tangent Command when the spline is reconstructed.

I added an auxData parameter to the StrokeDataVO, which allows arbitrary auxiliary data to be recorded for a stroke.  It’s easy to deep-copy an Object with a ByteArray, so preserving immutability was no problem.  Now, by adding support for a ‘redraw’ parameter in the PolyLine stroke, the spline can be redrawn with arbitrary tangents supplied by external data instead of the tangents computed by the injected tangent Command.

FDL Cubic Bezier Spline

The cubic bezier spline drawing engine in the Freehand Drawing Library is now complete.  As I mentioned in the prior post, the drawing engine fits into the FDL architecture and allows a spline to be treated as a stroke.  The CubicBezierSpline drawing engine contains a reference to a FDLCubicBezierSpline, which handles all the spline computations except for tangents.  Tangents are implemented via a Command pattern (with the spline as the receiver of the Command).  Three tangent commands are now available for cubic splines.  The NormalBisector Command uses the same algorithm as described in this TechNote.  The CatmullRom Command uses an algorithm similar to that in Catmull-Rom splines.  The Corner Command can be applied on a per-vertex basis and constructs splines with hard ‘corners’ or only G-0 continuity at a join point.

While the CubicBezierSpline engine manages the internal spline and tangent commands, the spline and tangent computations can be easily used outside the FDL in a purely computational context.

Every FDL spline is arc-length parameterized by default, using a fast algorithm for on-the-fly parameterization.  So, every coordinate request, i.e. getX(t) and getY(t) is actually computed based on normalized arc length in [0,1].  So, getX(0.5) returns the x-coordinate at approximately half the length along the spline.

Splines are implemented as drawing engines for a PolyLine stroke.  As always, a drawing engine is injected into the stroke,

__data.drawingEngine = “net.algorithmist.freehand.engine.CubicBezierSpline”;

Arbitrary parameters (name-value pairs) may be assigned to any drawing engine.  All splines require a ‘tangentCommand’ parameter to inject the Command used for tangent computations.  The normal bisector algorithm is applied as follows.

__data.engineParams  = {“tangentCommand”:”net.algorithmist.freehand.commands.NormalBisector”};

or, substitute the Catmull-Rom algorithm, if desired,

__data.engineParams  = {“tangentCommand”:”net.algorithmist.freehand.commands.CatmullRom”};

then, assign the data provider to the stroke,

__stroke.data = __data;

The cubic bezier spline supports auto-closure with G-1 continuity across the full set of tangent commands since the closure algorithm is inside the Command itself. Here is a screenshot,

and here is a screenshot of an open spline using the Catmull-Rom algorithm for tangent computations,

The next step is editing the spline by adjusting knots and tangents.  I was originally skeptical how far the architecture could be pushed in terms of a ‘drawing’ that was not in line with a traditional stroke motion, i.e. press-move-release.  Editing the mathematical properties of something like a spline seemed out of the question at first thought.  So far, I’m glad the architecture is holding up without having to hack anything in.  I suppose a demo containing a spline editor will be the ultimate test 🙂

As an aside, spline editing facilities are not part of the library; they are provided as one of the many demos available to FDL users.

Progress on Freehand Drawing Library Splines

Splines in the Freehand Drawing Library have been a challenge in two areas.  First, the concept of freehand drawing is centered around strokes.  Strokes are defined by sequence of mouse/touch points that begin with a press, continue with a sequence of moves (while pressing), and end with a release.  An interpolative spline is defined with a series of interpolation points or vertices and continuity conditions at the join points.  There is no concept of press-move-release. Fortunately, the FDL architecture is sufficiently general to allow points to be artificially added to strokes.  The UI issue with splines is creating a user-friendly mechanism for identifying the last interpolation point.

The second issue is editing, both vertex and in/out tangents at each vertex.  Since all strokes allow their input points to be either manually defined or cached and returned via user input, editing is not part of the core library.  It’s impossible to create an editing system that is general, maintainable, and equally satisfies all prospective users.  When it comes to splines, I don’t want to add more methods to the IFreehandDrawable interface simply to accommodate new features.  New methods should be added only with careful thought and because it makes good, long-term sense for the library.  Fortunately, ample back-doors exist to arbitrarily set and retrieve properties for various stroke engines through parameter access and mutation methods. I plan to use that existing scheme to expose parameters that facilitate spline editing.

Tangent construction is handled by a Command pattern that is applied across the engine spline or at individual vertices.  The tangent command is injectable and its fully qualified class name is part of the stroke engine parameter set.

There is currently one spline-based stroke engine for the PolyLine stroke.  The CubicBezierSpline engine is based on the cubic bezier spline in this TechNote.  The tangent construction is handled by a NormalBisector Command that is injected into the stroke engine.  I also plan to add another tangent command that uses the same approach as Catmull-Rom splines.  This provides two different cubic bezier splines with a single code base.  I also plan to provide a corner tangent command that allows sharp corners to be inserted into otherwise continuous splines.

One final feature of the architecture is the decoupling of the spline computations from the drawing facilities in the FDL.  This is similar to how I decoupled spline computations from the drawing architecture in Degrafa.  There are no actual spline computations performed in the CubicBezierSpline drawing engine.  It contains a FDLCubicBezierSpline, which is a concrete implementation of the IFDLInterpolatingSpline interface.  FDLCubicBezierSpline is completely independent of the FDL.

The final implementation issue is on-the-fly arc-length parameterization.  In order to draw the spline with line decorators, all requests for x-y coordinates along the spline are based on normalized arc length, not the spline’s natural parameter.  In other words, getX(0.5) returns an x-coordinate of a point that is approximately one-half the length along the entire spline.  The arc-length parameterization is optimized for monotonically increasing or decreasing normalized arc-length queries.  It presumes constant redraws of the spline, so there is no pre-computation/lookup applied in the parameterization.

A screenshot of work in progress is provided below.

During debugging, the outer edge of the convex hull of the bezier control points is drawn (in black).  The red dots are points on the cubic bezier spline at (approximate) uniform normalized arc length.  The next step is to apply a line decorator to actually draw the spline.  Another advantage of automatic arc-length parameterization is relatively easy computation of the Δs to use in point-to-point drawing of the spline.  This mimics a freehand curve like all the other engines, which allows automatic application of the existing line decorators in drawing the spline.

New Strokes in Freehand Drawing Library

Based on beta feedback, I’ve added two new strokes into the Freehand Drawing Library.  The first is an alternate variable-width stroke.  The standard variable-width drawing emulates a brush stroke that begins at negligible size and its width increases with speed (like pushing down harder on the brush).  The alternate stroke begins at a maximum size (initial point is a dot) and its width decreases with speed.  This is an experimental implementation and the creation as a separate stroke illustrates how to modify standard strokes for custom usage.  A screenshot is provided, below.

PolyLine is a constant-width stroke that interpolates a set of points.  An interesting feature of this stroke is that it is permanently non-interactive, that is it bypasses the normal stroke interaction for mouse or touch points.  Instead, the user manually adds vertices to the PolyLine.  Like any other constant-width stroke, it is assigned a solid-line drawing engine by default, which produces the drawing below.

The dots are manually added by the demo program.  In the future, they will be draggable.  This demo is currently a placeholder to test the spline drawing engines, now under development.

Because of these new additions, I’ve opened up several more beta slots.  There are currently two remaining.

The FDL is a serious commercial product, designed for serious commercial applications.  All FDL distributions include source code with full ASDoc.  As such, a small three-figure license fee is required to join the beta.  Beta users, however, get free upgrades for the life of the product.

If you have an application that could benefit from the FDL, especially if it is mobile or involves spline drawing, please contact me at theAlgorithmist [at] gmail [dot] com to discuss joining the beta program.

Quadratic Arc Decorators for Freehand Drawing Library

I just finished the first cut at the quadratic arc decorators for the Freehand Drawing Library.  Previously, only solid arcs could be drawn.  The example below shows dashed- and dotted-line decoration applied to arrowed arcs.

The decorators all share a fast, on-the-fly arc-length parameterization utility.  I still have a few tweaks to make to one of the decorators before full production release, but the new library is going out to all beta users this morning.

Based on feedback, I’m going to add one more stroke (Polygonal) and two spline-based drawing engines to the full release.  Because of this new addition, I’m seeking a couple more beta users.  If your drawing requirements could benefit from a variety of splines, including splines that auto-convert to shapes (i.e. lines and quads) for creating dynamic outlines/fills, please e-mail me for more information on entering the beta program.

The beta includes full source code and there is a small, 3-figure license fee.  All beta users get FREE upgrades for life.  Email is theAlgorithmist [at] gmail [dot] com.  Thanks!

Graphing Non-Functions

My current gig is winding down, so this will probably be the last update on the function graphing engine for a while.  Although the function graphing engine is architected to graph cartesian or parameteric functions, only cartesian functions were supported in the initial release.  All functions defined in XML must implement an IPlottable interface, which means that evaluation and derivative methods are interpreted with y as a strict function of x.  All functions must be marked as cartesian, since a parametric function will not plot in the initial implementation.

What about non-functions, that is cases where y is not strictly a function of x?  As the saying goes, there is always a back door.  Functions may identify themselves as self-plotting.  In this case, the engine does not sample the function.  Instead, it provides the function with all the relevant information regarding the current graph window and calls the function’s plot() method.  A self-plotting function is welcome to draw anything it pleases, including Bezier curves, conic sections, spirals, or even smiley faces 🙂

Problems may arise with derived functions, however, i.e. other functions that derive their visual display from the definition of another function.  A graphical Marker function is a perfect example.  The Marker is an artist-generated, interactive graphic asset that is constrained to the path of another function.  If domain values exist for which there is not a unique range value, then Marker movement is unpredictable.

This example shows how to work around the non-function issue to a degree.  It plots parabolas that open upward, downward, and to the left or right.  More specifically, the parabola directrix is either parallel to the x or y axis.  Parallel to the x-axis is good.  Parallel to the y-axis causes problems.

Suppose we want a Marker constrained to the parabola along with a display of the tangent and normal, as well as the directrix.  The fixed distance from the point on the parabola to the directrix and axis of symmetry is also drawn.

Of the four use cases, the Marker follows y as a function of x in two and x as a function of y in the other two.  We can’t use the function graphing engine to manage the Marker display as it always calls the base function (parabola) eval() method with an input x-coordinate.  The derivative and normal displays won’t work in two cases because dy/dx is not uniquely defined.

In general, the equation of a parabola with vertex (h,k), focus (h,k+p), and directrix y=k-p is given by

(x-h)2 = 4p(y-k)

which allows y to be defined as a function of x, or x as a function of y.  There are really only two use cases requiring consideration, directrix parallel to the y-axis and directrix parallel to the x-axis.  In one case, horizontal mouse movement is used to position the Marker.  Vertical mouse movements control the Marker in the other case.

The custom Marker is composed directly into the custom Parabola function, which is marked as self-plotting.  This function is responsible for creating the Marker and controlling its placement as well as drawing all supporting graphics.

The Parabola function is defined in XML as

<function id=”Parabola” class=”graph.tests.Parabola” params=”h:0,k:0,p:-1,parallelTo:y”>
<lineMetrics thickness=”2″ color=”0xff0000″ />
<data markerParams=”marker:graph.symbols.CustomMarkerSymbol,rolloverColor:0x9ACD32,digits:2″
markerCoord=”1″ >
<directrix thickness=”3″ color=”0x9933CC” alpha=”1″ lineStyle=”line_dashed” dashWidth=”6″ dashSpacing=”4″ />
<lineSet1 thickness=”2″ color=”0x0000ff” />
<lineSet2 thickness=”1″ color=”0x00ff00″ alpha=”0.5″/>
</data>
</function>

and the display is shown below.

The aspect ratio of this graph is not ideal for the example, but it’s adequate for illustrating the example.  Note that the custom Parabola function is welcome to plot not only the parabola, but as many supporting graphics as it likes.  The custom Marker may be dragged along the boundaries of the parabola and rollover displays the current y-coordinate by default.

A simple parameter change,  h:0,k:0,p:1,parallelTo:y, causes the parabola to open to the right.

Parametric function graphing should be supported in the future, providing for a more clean and general-purpose means to plot any general parabola and supporting visuals.  I thought this was an interesting example of how a decent architecture and simple concepts like Composition open up possibilities that do not seem possible based on the defined constraints of an engine.

Next post will be an update on new capability in the Freehand Drawing Library.