SKILL for the Skilled: Many Ways to Sum a List (Part 8): Closures -- Functions with State

In the past several postings to this blog, we've looked at various ways to sum a given list of numbers. In this posting I'll present yet another way to do this. This time the technique will be markedly different than the previous ways, and will take advantage of a powerful feature of SKILL++, namely lexical closures. These closures will be used to implement data encapsulation, and we'll also use lexical closures to capture computation state and continue the computation later.

Put the CIWindow into SKILL++ mode

Before proceeding, we need to change the listening mode of the CIWindow. We would like the CIWindow to interpret input expressions as SKILL++ rather than traditional SKILL.

Normally, when you type SKILL expressions into the CIWindow that defines functions or defines variables, the semantics of your code is taken as traditional SKILL. If, however, you would like to have SKILL++ (Scheme) semantics, you can put the CIWindow into SKILL++ Mode by calling the function (toplevel 'ils). This function does not return immediately, but rather puts the CIWindow into a different listening mode until you call the function resume, causing the (toplevel 'ils) to return.

You can find out which listening mode the CIWindow is in either by looking at the indicate in the button left-hand corner of the CIW. If in SKILL listening mode > will be inconspicuously displayed. The > is a little difficult to notice because it is so inconspicuous.

However, if in SKILL++ listening mode ILS-> will be displayed.

You may also determine the listening mode by calling the function theEnvironment which will return either nil if in SKILL mode, or non-nil, if in SKILL++ mode.

 

(theEnvironment)  ==> nil
(toplevel 'ils)
(theEnvironment)  ==> envobj@0x18fa2020
(resume)  ==> nil
(theEnvironment)  ==> nil

Defining an adder

With the CIWindow in SKILL++ mode we can proceed to define a SKILL++ function.

(unless (theEnvironment)
   (toplevel 'ils))

(defun make_adder_8a ()   ; 1.1
  (let ((sum 0))          ; 1.2
    (lambda (n)           ; 1.3
      sum = sum + n)))    ; 1.4

This definition of make_adder_8a is a 4 line function, yet does a lot in its 4 lines. It is a higher-order function, as seen in SKILL for the Skilled: What is SKILL++? it is a function which returns another function. It is a function which creates and returns a special kind of function called a lexical closure. In particular the lambda on line 1.3 creates a unary function which when called will evaluate the expression on line 1.4. However, the expression on line 1.4, references a variable, sum defined on line 1.2 and which is external to the lambda. In this case sum is called a free variable.

The important feature of SKILL++ which makes this magic work is that when make_adder_8a gets called, the let on line 1.2 creates a new binding for the variable named sum. A binding is a newly allocated memory location which is associated with a named variable. The initial value stored in this memory location is 0 as indicated on line 1.2. SKILL++ then proceeds to line 1.3 where it creates an anonymous function, attaching it to this binding. The two occurrences of the name, sum, on line 1.4 (within this anonymous function) are compiled as references to the binding.

The make_adder_8a function returns the anonymous function object created on line 1.3, without evaluating the code on line 1.4. Critically this function object maintains a link to the sum binding. Thereafter when the anonymous is called, the expression on line 1.4 is evaluated and its value returned. In evaluating this expression, the value of sum (in this allocated binding) is referenced and updated by the expression sum = sum + n, n being the value passed to the anonymous function when it is called.

Testing make_adder_8a

Let's experiment with make_adder_8a. Keep in mind that make_adder_8a does not actually add anything, rather it creates a function which is capabile of adding.

A = (make_adder_8a)       ; 2.1  ==> funobj:A
(A 1)                     ; 2.2  ==> 1
(A 2)                     ; 2.3  ==> 3
(A 3)                     ; 2.4  ==> 6
(A 4)                     ; 2.5  ==> 10
(A 5)                     ; 2.6  ==> 15

Arduous line-by-line explanation

On line 2.1, make_adder_8a is called, and as described above a SKILL++ anonymous function object (a closure) is created and returned. This closure is stored in the global variable A.

The CIWindow prints this value as funobj:A. As mentioned before the initial value of sum is 0. Note that sum is not a global variable. Rather it is a variable which is visible only to the code on lines 1.3 and 1.4. Furthermore, notice that we have no immediate access to the variable sum. We cannot query the value of code and we cannot modify its value, except by calling the function we have just stored in the global variable A.

When line 2.2, (A 1) is evaluated, the expression on line 1.4 is evaluated: sum = sum + n with n=1; sum is updated from 0 to 1, and 1 is returned and printed into the CIWindow.

When line 2.3, (A 2) is evaluated, again the expression on line 1.4 is evaluated: sum = sum + n with n=2. This time, sum is updated from 1 to 3 and 3 is returned.

When lines 2.4, 2.5, and 2.6 are evaluated, sum = sum + n is evaluated three times with n=3, n=4, and n=5 respectively; thus sum is updated to 6, 10, and finally to 15.

Summing a list with an adder

You can use an adder as created by make_adder_8a to add the elements of a list incrementally.

A = (make_adder_8a)          ; 3.1  ==> funobj:A
(mapcar A '(1 2 3 4 5))      ; 3.2  ==> (1 3 6 10 15)

This call to mapcar iterates across the given list (1 2 3 4 5) and calls the function A on each iteration, each time with the successive element of the list. Since each call to A returns the current value of the partial sum, mapcar returns not he final sum, but rather the list of partial sums computed at each step of the iteration.

  

Multiple SKILL++ adders in parallel

You can create several adders which work independent of each other. In the following example, we create two adders, A and B. Each one internally maintains its own partial sum of the arguments given to successive calls.

B1 = (make_adder_8a)       ; 4.1  ==> funobj:B1
B2 = (make_adder_8a)       ; 4.2  ==> funobj:B2
B3 = (make_adder_8a)       ; 4.3  ==> funobj:B3
(B1 1)                     ; 4.4  ==> 1
(B2 10)                    ; 4.5  ==> 10
(B3 100)                   ; 4.6  ==> 100
(B1 2)                     ; 4.7  ==> 3
(B2 20)                    ; 4.8  ==> 30
(B3 200)                   ; 4.9  ==> 300
(B1 3)                     ; 4.10  ==> 6
(B2 3)                     ; 4.11  ==> 60
(B3 30)                    ; 4.12  ==> 600

This works because each call to make_adder_8a on lines 4.1, 4.2, and 4.3, each allocate a new closure (three in all), assign each of them respectively turn to the global variables B1, B2, andB3. Each of these three function has its own independent binding of sum, each of which is initialized to code. When lines 4.4, 4.7, and 4.10 are evaluated, the sum binding of B1 is updated, but the sum bindings of B2 and B3 are not effected. Similarly when 4.5, 4.8, and 4.11 are evaluated, the sum binding of B2 is effected. And similarly for lines 4.6, 4.9, and 4.12.

Using flet as an alternative to lambda

If you find the use of (lambda ...) to be confusing in the definition of make_adder_8a, you can, as an alternative, define the same functionality using flet.

 

(defun make_adder_8b ()       ; 5.1
  (let ((sum 0))              ; 5.2
    (flet ((add (n)           ; 4.3
             sum = sum + n))  ; 5.4
      add)))                  ; 5.5

This implementation of make_adder_8b uses flet to define a local function named add. The normal pattern of using flet which you've seen in previous posts of SKILL for the Skilled such as Continued Introduction to SKILL++, is to define a local function and call it. The pattern used by make_adder_8b is to define a local function and return it, allowing the function which called make_adder_8b to call it, or likewise return it to its caller.

Data Encapsulation and Object Orientation

Some programming languages present the ability to encapsulate data as part of the object model. In these languages private variables are member variables within classes, and methods in/on those classes awkwardly manipulate and access these private variables.

This unholy marriage of private data to object model is limiting in practice because it is not only object oriented programs which need to hide data. In fact, a program written in any style may need to take advantage of encapsulate. In SKILL++ (and other dialects of Scheme), data encapsulation is independent from the object system, as it should be.

In SKILL++ the let and lambda primitives behave differently than in traditional SKILL. They behave in a way which allows a function to create lexical closures. These lexical closures are then able to manipulate the state of private variables which they encapsulate.

Summary

In this article, we looked at how to use lexical closures which maintain their internal state to implement counters. We looked very lightly and abstractly into how this is implemented within the SKILL++. And we traced step by step though a couple of examples of how this works in practice.

In this way SKILL++ provides data encapsulation completely independent from the object system. While SKILL++ does indeed have an extensive, full-featured, and powerful object system, in SKILL++ you are not forced to write a program in an object oriented way just to encapsulate/hide data. Encapsulation is a feature of SKILL++ which is available to you whether you are using OO, imperative, declarative, functional, or any other style you choose.

More to come

In the next post we'll look more at the differences you'll see when defining these types of functions in SKILL++ vs in SKILL.

See Also

SKILL for the Skilled: What is SKILL++?
as Continued Introduction to SKILL++

Jim Newton