This is a frequent question I receive from those wishing to parlay a knowledge of Javascript into a long career as a developer. Speaking of long careers, mine currently spans 35 years of professional experience. I began this long journey writing assembly-language math libraries for supercomputers. Of course, how you start is rarely indicative of how you will end. Since then, I’ve also worked with Fortran, C, C++, Java, JavaScript (and Mocha/LiveScript), ActionScript (1/2/3), and Typescript. This reminds me of the line from RUSH’s popular Tom Sawyer song, “changes aren’t permanent, but change is.”
Whenever I take on the task of learning a new language, I’m always interested in the internals of what makes that language work. What was the design philosophy of the language? Is there a single concept or a group of core concepts that explain how and why the language/environment produces run-time results? I’ll discuss three areas in this post and then provide a second post on additional topics if there is sufficient interest.
Functions are first-class language objects in Javascript and almost everything of importance in an application revolves around them. So, I’ve personally found understanding function execution context to be highly valuable in constructing an understanding of Javascript internals.
EXECUTION CONTEXT
Now, I don’t like re-inventing the wheel, so wherever possible, I will provide links to articles on individual topics that I’ve found to be not only accurate, but very helpful in leading to an in-depth understanding of that topic. And, when it comes to execution context, I still think Dave Shariff’s article is one of the best,
http://davidshariff.com/blog/what-is-the-execution-context-in-javascript/
This article is well worth spending a lot of time understanding, particularly the section on hoisting.
Following are a couple key takeaways from that article. You will hear the terms ‘context’ and ‘scope’ mentioned a lot and these concepts are not unique to JavaScript. A ‘scope’ is simply a well-defined area in a run-time environment that is used to look up variable references. Let’s not overcomplicate things. If I say to someone, “Hey, look at that red Mustang,” but there is a red Mustang in both the Kroger and Walmart parking lots, then I need to specify which lot (or scope) in order to make the vehicle (variable reference) unique. I may have to be even more specific and identify the red Mustang at the south end of the Kroger parking lot.
There are three possible contexts in Javascript,
– Global (the application has to start somewhere)
– Function (program flow enters a function body)
– Eval (execution inside an internal eval function)
So, what is ‘scope’ in Javascript? At any single point in the flow of program execution, it will be one of these three contexts (parking lots).
Execution Context consists of three items,
a) A scope chain, represented as an ordered list of variable objects (V0), or variable tables, that are ordered from the function’s direct parent back up to the global context.
b) An internal variable table or VO for the function itself that includes the arguments Array and an entry for each variable defined internally in the function.
c) A unique resolution to the self-referential pointer (i.e. ‘this’).
The final item requires some additional explanation. We must be able to uniquely and predictably resolve references such as this.x inside a function, so how Javascript assigns the self-referential pointer is very important. It is also unique to JavaScript in that the assignment depends on how the function is invoked. I’ve found this knowledge to be rather important as it can explain a lot when you are forced to debug/modify code written in the past by one or more devs.
Here are the ways JavaScript assigns the self-referential pointer based on function invocation:
1 – Method invocation. The function is a property of an Object. ‘this’ is bound to Object in which the function is a property.
2 – Function invocation. The function is defined outside an Object and invoked by direct call. ‘this’ is bound to the global (typically Window) object. Crockford correctly pointed this out as a flaw in the language and one that can result in undesired (and not easily understood) results. Let’s look at some examples.
Copy and paste the following code block into your browser console and execute,
var x = 2;
function foo() {
var x = 1;
console.log('x in foo:', this.x);
};
foo();
The function, foo, is defined ‘standalone.’ When it is called on the last line, that is known as Function invocation. The execution context for the foo function consists of a scope chain containing only the global context. The VO consists of a single variable, x, which is not defined until the activation stage of execution context. The arguments Array is empty. The self-referential pointer is set to the global context.
So, even though there is a reference to a variable named x inside the foo function, resolution of this.x comes from the global context. So, ‘x’ is logged as 2 in this example.
Suppose the self-referential pointer was removed and the example was,
var x = 2;
function foo() {
var x = 1;
console.log('x in foo:', x);
};
foo();
Based on what we have studied from the concept of execution context, x is defined inside foo’s VO as 1 during the activation stage of execution context. With only a ‘raw’ variable reference, the first attempt at name binding or variable lookup is from the function’s VO, from which a variable with that name exists. In this case, x is logged as 1.
Now, let’s change the example to
var x = 2;
function foo() {
console.log('x in foo:', x);
};
foo();
In this case, the first lookup for x inside foo’s VO fails. Subsequent lookups proceed along the scope chain defined during the activation stage of execution context. The first (and only) VO in this chain is the global context, in which the variable, x, exists. In this instance, x logs as 2.
And, since you understand that unique assignment of the self-referential pointer is part of a function’s execution context, and you understand the steps in variable lookup, you should understand that in this example,
var y = 2;
function foo() {
var x = 1;
console.log('x in foo:', this.x);
};
foo();
The variable, x, now logs as undefined as there is no such variable in the global context.
3 – Constructor invocation. The function is defined and stored in a variable and then invoked with the new keyword, i.e.
var x = 7;
var y = 8;
var Foo = function() {
this.x = 1;
this.y = 2;
};
Foo.prototype.z = function() {
return this.x + this.y;
};
var bar = new Foo();
console.log('z: ', bar.z());
This is an important example as it is the means by which traditional classes are approximated in JavaScript, which is built on prototypal inheritance, i.e. Objects simply inherit from other Objects. When constructor invocation is applied, an addition is made to the function’s VO that is a reference to the function’s prototype. The self-referential pointer is then set to that reference.
So, in the above example, the variable z logs as 3. However, in this example,
var x = 7;
var y = 8;
var Foo = function() {
var x = 1;
var y = 2;
};
Foo.prototype.z = function() {
return this.x + this.y;
};
var bar = new Foo();
console.log('z: ', bar.z());
z logs as NaN. Why? The function’s VO includes table entries for variables x and y, however, this operation does not call a method created by the function, but one that exists on its prototype. Variables x and y are never defined on the prototype, so the addition can not be coerced to a meaningful result (more on that later).
4 – Apply/bind – Apply/Call/Bind allow a function to receive a specific context for the self-referential pointer. This is the easiest invocation example to understand.
5 – Lexical propagation, aka arrow functions. Lexical propagation is simply a fancy name given to passing of the parent context of a function to the function itself. It’s simply a replacement for assigning a reference to the ‘this’ pointer to a variable, i.e. var parent = this or var that = this and then using the parent variable inside a child function body. Instead of this.x inside the function body, the code references parent.x, for example.
Lexical propagation via arrow functions allows a function to be defined independently using references such as this.x . The function may be imported into a Typescript class, for example, and lexical propagation provides an automatic method closure.
This statement leads us right into the next section 🙂
SCOPE AND CLOSURE
Well, now, we get into some of that computer-science stuff, but some principles are best understood from a more fundamental, language-independent context. Some of what you may read on these topics in Javascript are wrong or only partially correct.
Name-binding or the mapping of a name such as a variable to a specific value is important in any language. Scope is a specific region of the program in which a specific name-binding is applicable. Scope is normally identified at the syntax level, although it is applied at run-time.
Shariff’s article was written before ES6 and at that point in time, variables in Javascript were scoped at the function level. A variable x, for example, might be defined inside an if-block, but a property for that single variable is created inside the function’s VO during the activation stage of execution context. That same variable might be assigned a new value outside the if-block, but there is still only a single entry in the VO that serves to bind the name, x, to a single value inside the function, regardless of where the variable is declared inside the function.
ES6 introduced let and const variables that are block-scoped, which makes the function’s VO table more complicated. Name binding during the process of function execution can now depend on a specific, basic block inside the function body. This is illustrated by example,
var x = 2;
function foo() {
var x = 1;
var y = 2;
let z;
if (y > 0) {
let x = 3;
z = x + 2;
console.log('x/z inside if:', x, z);
}
console.log('x in foo:', x);
};
foo();
There are two references to the variable, x, inside the function, foo. So, how is name-binding applied? There are no references to the ‘this’ pointer, so the variable is looked up first inside the function’s VO table. However, there are two scopes in which x can be resolved. The first is block-level and is inside the basic block encapsulated by the if statement. Name-binding begins at the current scope, which is the if-block when the first console log is encountered. Since there is an immediate resolution of the variable reference, the value of 3 is logged for x. However, that scope only persists for the duration of the if-block. If the let x = 3 statement were removed (try this) then name-binding is applied at the next-highest scope, which is the function level. There is a match at that level, so x logs as 1. If the var x = 1 statement were removed, there is no match anywhere in the function’s VO, so lookup proceeds along the scope chain created during the activation phase. As with the other examples, there is only one VO in that chain, which is the the global context. There is a match on x in that VO and the value of 2 is applied to the variable, x.
So, in this example, we could say that there are three scopes or regions of the program in which name-binding can be applied, each of which produce a possibly different resolution for the same variable name, or the variable may not be bound at all. These scopes are the if-block inside the function foo, the function itself, and the global context. Think of it as dividing the parking lot into north and south partitions and then further specifying an east/west partition for the south partition. These union of these partitions may or may not be the entire parking lot, so we may think of the entire parking lot as the global context. The variable ‘red Mustang’ is uniquely resolved (or not resolved at all) based on a specification of scope.
Now, what about those pesky closures you may have heard of or read about somewhere else? Yes, here we go with more computer science stuff …
I like the Wikipedia definition of closure the best, which is simply an implementation of lexically scoped name binding in languages with first-class functions. Lexical scoping refers to independence from a run-time environment and dependence upon the lexical or program-level structure of the language to define scope. A closure is applied to a function as an independent entity by creating a combination of the function with a context for name-binding. From the prior material, we can see that a function closure is created by the totality of the environment created during execution context.
When closures are applied to a method of a ‘class’ (recall this method exists on the function’s prototype), this combination of function and context is called a method closure. ActionScript, for example, was ECMAScript compliant since version 5 of the Flash player. ActionScript 2 introduced the syntactic sugar of classes that compiled down to AS 1 in a similar manner to how TypeScript compiles down to ES5. Classes are not first-class language objects in any current version of JavaScript and this was also true of AS 2. To use a class method as an event handler, for example, we created method closures using a helper function called Delegate.
For the remainder of this article, we can infer that closure always means function closure. Closures may be created and manipulated by developers, generally for the purpose of variable privacy or to avoid undesirable side-effects of code that may seem to perform a required task, but do not execute as desired. Closures may also be used in the technique of partial application of functions and the following is a good introduction to the topic,
https://medium.com/javascript-scene/master-the-javascript-interview-what-is-a-closure-b2f0d2152b36
While you can understand a lot (theoretically) about closures through function execution context, there are some subtle issues with how they are created and applied that are best illustrated by example. Consider the construction-invocation example, above, with an expanded console log.
var x = 7; var y = 8; var Foo = function() { this.x = 1; this.y = 2; }; Foo.prototype.z = function() { return this.x + this.y; }; var bar = new Foo(); console.log('z: ', bar.z(), bar.x, bar.y);
Execute this inside your browser console and note that z, x, and y, log to 3, 1, and 2, respectively. While it may be tempting to consider the variables x and y to be ‘closed’ inside Foo, they are completely visible (and can be manipulated) outside that function,
var x = 7;
var y = 8;
var Foo = function() {
this.x = 1;
this.y = 2;
};
Foo.prototype.z = function() {
return this.x + this.y;
};
var bar = new Foo();
bar.x = 10;
bar.y = 11;
console.log('z: ', bar.z());
Ouch. Now, z is 21.
While the latest version of JavaScript supports truly private variables in ‘classes’ by prefixing with # symbol, you may be called upon to work with code created by other developers, perhaps many years ago, in which manual closures were created to facilitate data privacy. For example,
const point = function(x, y) {
let xCoord = x;
let yCoord = y;
return {
getX: () => xCoord,
getY: () => yCoord
};
};
const somePoint = point(0, 3);
const xValue = somePoint.getX();
const yValue = somePoint.getY();
console.log('values:', xValue, yValue, somePoint.xCoord, somePoint.yCoord);
Note that this logs 0, 3, undefined, undefined.
The accessor functions, getX() and getY() reference variables defined in the VO of the point function, although we can not directly access those variables from outside the point function. This is unlike the prior example, which utilized the Foo function’s prototype (which any Object may access) to create variables and methods. Sometimes, functions such as getX() and getY() are referred to as privileged methods. These methods were defined at a level in which name-binding of the variables xCoord and yCoord could be resolved to known, defined values. So, they are ‘privileged’ to have access to that information during their own execution context activation stages as a result of being defined inside the function, point.
This type of closure can also be used to create stateful functions, sometimes in tandem with an IIFE (Immediately Invoked Function Expression). An example I use when teaching this topic is a Wythoff iterator. Strictly speaking, this iterator does not produce sequences that are exact rows of a Wythoff array, but it is Fibonacci-like and accepts two starting values, just like the Fibonacci sequence. Subsequent values of the iterator are obtained by adding the two previous values. This example, with starting values of 0 and 1, could be considered a Fibonacci iterator. Now, the Fibonacci sequence is a recurrence relation with constant coefficients and all such relations have a closed-form solution (see Knuth TAOCP). The closed-form solution for the Fibonacci sequence is Binet’s formula, but that is not used in this example. Instead, we maintain internal variables, first and second, which are overwritten each time the iterator’s next() function is called. This code is crude, but serves to illustrate the desired concept,
const iterator = (function (f, s){
let first = f;
let second = s;
return {
next: () => {
const tmp = first + second;
first = second;
second = tmp;
return tmp;
}
}
})(0, 1);
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next());
console.log('next:', iterator.next(), iterator.first, iterator.second);
The IFFE results in immediate invocation of the function with the arguments 0 and 1. This triggers the activation and execution phases of that function’s execution context in one step, so to speak. Space for the function variables first and second is created in the VO table during the activation phase. Specific values of 0 and 1 are assigned to those variables in the execution phase. Since the result is stored in a variable, iterator, we might think of this process as defining a state for the function that influences its execution on each call of the privileged method, next().
Run this code in a fresh browser console and you should see the result,
next: 1 next: 2 next: 3 next: 5 next: 8 next: 13 next: 21 next: 34 next: 55 undefined undefined
The final line once again illustrates that we can not directly access the internal variables, first and second. State is maintained between function calls.
This example can be extended more complex examples and many have labeled this general approach the ‘Revealing Module Pattern,’
Loop induction variables that serve as function arguments provide another interesting example of closures and a segue into some of the newer language features. Consider a classic example,
var funcRefs = []; for (var i = 0; i < 5; i++) { funcRefs[i] = function() { console.log("value is: " + i); }; } for (var j = 0; j < 5; j++) { funcRefs[j](); }
Run this in a browser console and you will see ‘value is: 5’ repeated five times. The variable, i, is scoped to the global context in this example, or a level just above the first for-loop. That variable is maintained after loop execution is terminated and its value is 5. Why 5? The induction variable is incremented before the test for loop termination is made.
When each anonymous function is created, its scope chain contains the global context as part of the execution context created during the activation phase. When each function is executed, variable lookup starts at the anonymous function’s VO. There is no reference to a variable, i, so lookup proceeds up the scope chain. That name is bound in the global context to the fixed value, 5. So, in the second loop, each function resolved the variable, i, to the same value.
I’ve seen a number of resolutions to this problem, even to the extent of IIFE’s created inside the loop. I’ll show you the ‘classic’ or ‘old-school’ resolution to the problem and then discuss a couple more modern solutions, should you encounter this case in code you are assigned to debug and/or modify.
From the example above, we can use the internal VO table of a function as a means to store ‘state.’ The value of the induction variable for the first loop can be passed to a function factory and then stored in the VO for that function,
var funcRefs = []; function fcnFactory(i) { return function() { console.log("value is: " + i); }; } for (var i = 0; i < 5; i++) { funcRefs[i] = fcnFactory(i); } console.log('variable i is:', i); for (var j = 0; j < 5; j++) { funcRefs[j](); }
Verify that this displays the result that was originally intended,
variable i is: 5 value is: 0 value is: 1 value is: 2 value is: 3 value is: 4
The problem in this example is function-level scoping. If we could make the first loop induction variable block-scoped (as in ES6), then the example should work as desired. This is due to the fact that when the anonymous function is created, the ‘outer’ scope is the context containing the value of the induction variable at each loop iteration.
var funcRefs = []; for (let i = 0; i < 5; i++) { funcRefs[i] = function() { console.log("value is: " + i); }; } console.log('variable i is:', i); for (var j = 0; j < 5; j++) { funcRefs[j](); }
Note that in this example, the loop induction variable does not exist outside the loop; it can only be bound inside the loop and each iteration produces a different bound value.
We can also avoid problems of this nature with automatic closures provided by Array methods such as forEach. Each invocation of the callback function is given its own context (or closure) based on the variable(s) used in the function argument(s).
COERCIONS
This is my final must-know topic in part I of this series, especially since JavaScript likes to ‘make things work,’ even things that on the surface seem to make no sense. In order to better understand what I’m talking about, let’s discuss the difference between casts and coercions.
A cast in any language is a ‘hard’ statement that is intended to force the compiler to generate code to convert a variable of one type into a variable of another type. In C++, for example, a developer may force a variable to be converted from a double to an int by a statement of the form,
n = (int) z
where the variable, n, is typed int and the variable, z is typed double. This statement could be interpreted as “yes, I know z is a double-precision float, but FORCE it to be an integer and then assign the result to the variable, n“. Casts, while specific, still require some understanding of how the cast is performed. For the above example, does the compiler truncate or round the floating-point number?
At least a cast is a specific statement by the developer that indicates intent. A coercion on the other hand is performed silently by the environment and some developers may not even know that the coercion occurred. Coercions occur when assignments are made from one variable type to another, during some operations, and even during comparisons. The latter case often takes new JS developers by surprise and is the primary reason that strict equality (triple-equals) tests are preferred over basic equality tests (double-equals).
Here is a great example. Study the table in this link and see what can happen when variables of different type are compared with a double-equals test.
https://dorey.github.io/JavaScript-Equality-Table/
Coercions occur ‘under the hood,’ sometimes without a developer understanding that they are happening at all. Internally, JavaScript ties to convert one or more variables of an existing type into a type that somehow makes the operation or assignment ‘work,’ even if the definition of ‘work’ does not make sense to us. There are also some instances in the language that are somewhat in-between a cast and a pure coercion.
One example is the infamous double negation, ie. var x = !!y. There is no double-negate operator and I personally do not like this practice. But, some people like writing tricky or excessively compact code, so you may encounter this syntax in pre-existing code. The first negation attempts to convert the existing variable, y, to a Boolean primitive and then logically negates the result. Something that is truthy is converted to Boolean false and something falsey is converted to Boolean true.
The second value logically negates the first, so the final result of the coercion is that truthy variables are converted to Boolean true and falsey variables are converted to Boolean false.
Another example is placing a plus or minus sign before a string variable, i.e. +x or -y (the unary prefix operator). This operator attempts to coerce the string into a positive or negative number.
JavaScript actually has a very organized system for converting complex objects to primitives and then converting from one primitive to another. I remember an discussion with a somewhat new JS developer (less than five years experience) who was frustrated in an interview when asked to provide the result of the operation [] + {}.
I think the question, as posed, is a bit ridiculous as it infers a sense that someone might intentionally attempt such an operation. It could, however, occur as a side-effect in a legacy system were variables were generalized from primitives such as string or number into Object and/or Array. These variables could be passed in some rare outlier case to a function that expected primitives, but received something more complex. JavaScript is in the business of results. Some result. Any result. Making sense does not matter 🙂
So, it does help to understand the internals of this coercion system, especially if you are called upon to work on code that has existed for a long time and has been revised by a number of developers over the years. Now, before we being, memorize the primitive types in Javascript, namely undefined, null, boolean, number, string, bigint, and symbol. Everything else is an Object (including Arrays and functions). Primitives are immutable and can not be changed, and we’ll discuss that more in part II.
On the subject of coercion, primitives do not have properties, so how is it that we can access the length property of a string? JavaScript will happily and temporarily coerce a primitive to an Object, imbue the temp Object with properties, and then make the Object available for garbage collection after the property access. Be very careful with the last sentence as it possible to assign properties to a primitive, but access to those properties is undefined. For example,
var someString = "Math is cool"; someString.numWords = 3; var words = someString.numWords; console.log(words);
The variable, words, is undefined. Try it in your console. Behind the scenes, JavaScript creates a temporary Object that allows assignment of the numWords property, but no permanent reference is created for that Object. It is immediately marked for garbage collection. JavaScript creates a second, new temporary Object in order to allow access of the numWords property, which of course, does not exist.
Now, how does Javascript perform coercions from Objects to primitives and primitive to primitive? In particular, how does JavaScript perform seemingly crazy operations such as [] + {}?
I’ve found this article to be an excellent introduction to the topic,
https://2ality.com/2012/01/object-plus-object.html
The key takeaways from this article are first understanding important built-in Object methods such as valueOf and toString. Then, the coercion process can be broken down to convert to primitive followed by convert to similar primitives. The answer to the above question is covered in that article, namely the string, ‘[object Object]’.
Coercions are subtle because there is no direct indication in the code (such as a hard cast) as to their application. Even casts may not produce a desired result simply because at run-time, JavaScript attempts to perform the cast and then executes a coercion before completing execution of a specific line of code. There is only one way to become effective a dealing with coercions and that is to study JS internals and understand where coercions are applied. That will make it easier to diagnose issues in legacy code and, hopefully, force you to take more time in being explicit about intent when writing your own code.
Well, that brings us to the end of part I and we have not discussed important topics such as event loop, pass by value or reference, data structures such as Set/Map, Promises, and more. If there is interest, I will author parts II and III of this series an we can dive into those topics along with curated links to quality content that should be part of your regular study.
In the mean time, here’s a preview of what is coming next. Try predicting and then explaining the output of the following code block. I regularly use this example in lectures to JS students.
var arr = [1, 2, 3];
var value = 1.0;
var somefunction = function(v, a)
{
v = 2.0;
a = [4, 5, 6];
};
somefunction(value, arr);
console.log( "v/a: ", value, arr);
var anotherfunction = function(v, a)
{
v = 3.0;
a[0] = 4;
a[1] = 5;
a[2] = 6;
};
anotherfunction(value, arr);
console.log( "v/a: ", value, arr);
Hint: Primitives are immutable and are passed by value. JavaScript creates a separate handle or reference to Objects and passes that reference by value. I like to use the analogy of a pointer variable in C and C++ . So, there is a difference when a pointer’s value changes and thus points to something new vs. dereferencing that pointer. The copy still points back to the original entity, so the dereference affects the original’s value. Okay, that was more like an explanation than a hint, but I hope it stokes your appetite for new content.
Thanks for reading such a long post and best of luck to you in your JavaScript career!
The Algorithmist 