Hash sign

I was looking at the animation example…it has the line:

parameter.integer("TestNumber", 1, #allTests, 1, setTest)

What does #allTests do? I mean the hash sign…I can’t find any info on it anywhere…

returning the count of entries (=number of: children/entries/key=>value pairs). You can use this only with tables and keys have to be numerical.

Hello @david19801. Lua’s length operator (hash) is explained here.

Is it the same as table.maxn? or different?

table.maxn returns the largest positive numeric index of the table, even if the indices aren’t contiguous. The #table operator will return the largest contiguous numeric index of the table (so if there is a gap in your indices, it will ignore it).

I have a slightly different answer: for tables for which the result of the length operator (#) is defined (update: in Lua 5.2), # and table.maxn return the same value; for other tables, it depends.


function setup()    
    t = {10, 20, 30, nil, 50} -- Not a sequence, length operator not defined
    print(#t) -- Output: 5
    t["x"] = nil
    print(#t) -- Output: 3
end

function draw() background(0) end

This draws on the updated part of the Lua 5.2 reference manual that (update: in terms of the underlying implementation) is equally applicable to Lua 5.1 (here).

The length operator is not defined (update: in Lua 5.2) for tables that are not a sequence. A table is a sequence if the set of its numerical keys is equal to {1…n} for some positive integer n.

I was basing my thoughts on the following difference in output:

t = {}
t[1] = "x"
t[2] = "y"
t[3] = "z"
t[5] = "w"

print( #t ) -- Outputs: 3
print( table.maxn(t) ) -- Outputs: 5

The length operator isn’t defined in this case, but it seems to return the largest contiguous index starting from 1, where maxn simply returns the largest numerical index.

To follow up with Simeon’s post, I think something like this is what we all initially expect to see for the result:

  n = 0 
  for x,y in pairs(t) do
      n = n + 1
  end
  print(n) -- outputs: 4

Is there an operator or function in lua that does this?

Unfortunately no, @Ceres. It’s one of the criticisms of Lua that the table structure doesn’t maintain a count of its elements.

the # operator on tables is clearly defined, it’s the tables that are unusual. Tables in lua are always hash tables (we ignore implementation details for now), so basically, from the tables’ point of view, placing stuff at index 1 or index “one” does not make much of a difference, apart from being different indices of course :wink: The implementation does however offer the possibility to treat tables as arrays. And this is where the confusion about # comes from. Basically, the # operator on a table t returns any one numeric element n, for which t[n] ~= nil and t[n+1] == nil (with a special case if t[1] is nil, in which case #t can return 0). Note: this is not necessarily the first or last such element, just any one element for which this condition is true.

Because of what tables are, finding a definition for # that works and does what everyone expects from it, and is fast, is nigh impossible. Consider this:

t = {}
t[1] = true
t[500] = true
t[1000] = true

now, what should #t return in this case? With the current definition it could return either 2, 501 or 1001, this is useful in a sense that you can place an element at the returned position without overwriting another element, and maintaining consecutive indices. This is why a common (and recommended) idiom for appending stuff to array like tables is to use t[#t+1] = x. If it returned the number of elements in the table, it would return 3. Now, while for hash tables this might be useful, when used as arrays, this is completely useless… So, how about # returns the highest numeric index in the table whose value is not nil? In this case it would return 1000, which would suit most people. But, in this case, what would happen, if we set t[1000] to nil? On the next call to #, lua would have to scan the entire table in order to find the next smaller element for which this condition holds. Note that this is what table.maxn does, so beware!

So, what is the conclusion from this? Basically it is, if you want # to return the length of a lua table used as array, don’t have nil’s in the table. nil denotes the absence of a value, and if you set a table value to nil, lua removes the entire key-value pair. If you need a value that tells you that a field is empty, create one. A good way to do this is to just create an empty table, which by its nature will different from all other tables, and as such unique. The flipside of this is, that in the example above you would have to initialize the table with this empty value for all fields between 1 and 1000. But then, this is a rather artificial case. If you want something else, look into the __newindex and __len metamethods at http://www.lua.org/manual/5.1/manual.html .

All of this becomes even more interesting, if you consider that a table can hold both numeric and non-numeric keys (or even fractional numbers, which are not all that useful, tough, because of the properties of floating point arithmetic on computers…). What should # do in those cases?

This is really a quite complicated topic, and has been discussed on the lua list many times. All things considered, I feel that the current implementation of the # operator on tables is about as useful as can be done under such circumstances. Use caution, and you’re ok, and for the other cases, lua does provide you with the mechanisms to roll your own.

@gunnar_z do you know Lua doesn’t associate a count with its hash tables? If that were the case, the # operator could simply return the total number of elements in the table (irrespective of numeric or contiguous indices).

@Simeon yes I know that, but returning the number of elements in a table from the # operator is bound to cause confusion, too… especially with only numeric keys, and when the indices are not consecutive. Or worse yet, tables with numeric and non-numeric keys…

Anyways, people have different preferences on what # should return. From the recurring posts on lua-l I get the impression that people mostly expect # to return what table.maxn returns, which, for performance reasons, would not be such a good idea. I just outlined why things are as they are, and took the liberty to state my preference ;)As I wrote, there has been a lot of discussion about this.

Hello @gunnar_z. You cannot ‘rewrite’ (using its metatable) the length operator for table values in Lua 5.1 (used in Codea version 1.5.1). You can in Lua 5.2. table.maxn is deprecated in Lua 5.2.

Below, an explanation of my earlier example:


function setup()
    -- The constructor below creates a table with a array part of size 5
    -- and an empty hash part.   
    t = {10, 20, 30, nil, 50}
    print(table.maxn(t)) -- Output: 5
    -- t is not a sequence, so the length operator is not defined (in Lua 5.2 terms).
    -- As the array part is full and the hash part is empty, # will
    -- return the size of the array part:
    print(#t) -- Output: 5
    -- Adding a key to the empty hash part triggers a rehash (even one
    -- with nil as the value):
    t["x"] = nil
    -- After the rehash, the array part is of size 2^3 (8) and the
    -- hash part is empty. A binary search of the array part finds the
    -- boundary at 3:
    print(#t) -- Output: 3
    t[6] = 60
    t[7] = 70
    t[8] = 80
    print(table.maxn(t)) -- Output: 8
    -- The array part is full again, so # will return its size:
    print(#t) -- Output: 8
end

function draw() background(0) end

@mpilgrem you’re right of course, # can’t be overloaded in 5.1. You can create a length method, though, and use __newindex to find the length you care for. But the length operator is defined for your example, just not in the way you want it. As I wrote, it returns any one index so that the value stored at that index is non-nil and the value at index+1 is nil. It may not be deterministic, but it is defined. Maybe it is misleading to refer to it as the length operator in table context… but, I also mentioned the solution to your issue, if you use # as a length operator, don’t have “holes” in your table, and you’re all set.

I did not think this up, btw, I am just recounting what you might call common wisdom from lua-l. Back before lua 5 there was a field n in a table with integral keys. I think it contained the highest index set. This also did not turn out to be the one true solution to the length problem for reasons outlined in my post above… the whole table length issue does seem to not be solvable in a way that makes everybody happy with luas hybrid tables.

Hello @gunnar_z. The difference between us is only semantic. The Lua reference is said to be the “official” definition of the Lua language and the Lua 5.2 reference specifies (here) that the length operator is not defined for a table that is not a sequence. I understand ‘not defined’ in this context to mean that two table values that are logically identical can return different results when the operator is applied to them (as shown by my example).

@mpilgrem weeell… we’re on lua 5.1 here, and there it is defined. Actually, the wording in the 5.2 spec is a result of the confusion between people’s expectations about the # operator and what it really does. It kinda was agreed upon to call the nondeterministic behaviour in the case of tables with holes undefined.

But, honestly, we can end this here, because we don’t disagree. I you use a table as array, and have non-consecutive indices there, then # won’t make you happy.

.@gunnar_z I mistyped, I meant to ask: do you know why Lua doesn’t associate a count with its tables? Seems like an odd choice.

@simeon no, actually. I think (but this is just my interpretation) that it has to do with the lua team favoring iterators over a more direct approach of stepping through tables. But then, I also find it a bit weird that ipairs should also stop at the first unset index, instead of iterating over all numeric key-value pairs. Probably also has to do with performance, but seems counter intuitive.

.@gunnar_z On reflection, you are right. What defines the Lua 5.1 language is the corresponding reference, not the implementation. The underlying implementation of the length operator for table values (int luaH_getn (Table *t) in ltable.c) may not have changed between Lua 5.1 and Lua 5.2, but the definition did change.