What I wish they’d told me about metatables

It took a lot of reading to understand metatables.

Once I understood how metatables work I was a little irritated. It seemed like the explanations of them a) were too complicated and b) left out important things.

So I hope this helps anyone trying to figure them out.

TL/DR: __newindex interrupts writing to the main tables and __index interrupts reading from the main tables, but if you want them to work on all of a table’s read/write operations, you have to leave the main table completely empty and store all the values somewhere else.

Okay, so this explanation concerns the __newindex and __index metamethods.

The main thing it took me a while to understand is if a metamethod is triggered, by default it interrupts access to the main table..

And furthermore, if you want a metamethod to act upon every single call to a table’s indexes or keys, you must always leave the main table empty.

So first let’s look at a metatable that does absolutely nothing:

    emptyMeta = {}
    anyLittleOlTable = {}
    setmetatable(anyLittleOlTable, emptyMeta)
    anyLittleOlTable[1] = "gooba"
    print("anyLittleOlTable[1]: ", anyLittleOlTable[1]) --prints 'gooba'

You write to it, you read to it, just like normal. Now let’s add the the __newindex and __index metamethods, but without any code in them.

    emptyMeta.__index = function(tableBeingAccessed, key)
    emptyMeta.__newindex = function(tableBeingAccessed, key, newValue)

What these methods are going to do is interrupt access to empty indexes/keys. Trying to assign or read from indexes/keys that don’t already exist will route those calls to the metamethods, and since those methods are empty, nothing will happen:

    anyLittleOlTable[2] = "tuba" --index not used yet, so __newindex gets called, which does nothing, and value is not assigned
    print("anyLittleOlTable[2]: ", anyLittleOlTable[2]) --prints nil

But if the keys exist already, assigning and reading them will work as they would normally:

     anyLittleOlTable[1] = “Bradley gooba” --index in use already, so __new index not called, and value assigned normally 
     print("anyLittleOlTable[1] again: ", anyLittleOlTable[1]) --prints 'Bradley gooba'

But let’s say we do want the metatable to assign the value normally. Let’s make a new metatable and use rawset inside the metamethod (this is needed because, without it, assigning to an unassigned index/key would cause infinite recursion):

    usefulMeta = {}
    usefulMeta.__newindex = function(tableBeingAccessed, key, newValue)
        rawset(tableBeingAccessed, key, newValue)
    setmetatable(anyLittleOlTable, usefulMeta)

    anyLittleOlTable[2] = "Bradley tuba" --now __newindex passes value to main table
    print("anyLittleOlTable[2] again: ", anyLittleOlTable[2]) --prints 'Bradley tuba'

Okay now let’s make the __newindex function actually do something new. It will turn the value into a string and add “better than ever” to it:

    usefulMeta.__newindex = function(tableBeingAccessed, key, newValue)
        local adjustedValue = tostring(newValue).." better than ever"
        rawset(tableBeingAccessed, key, adjustedValue)

So, again, this will only be triggered when a value is assigned to an unused key or index:

    anyLittleOlTable[3] = 44445 --key/index is not used already, so __newindex adjusts value and stores it
    print("anyLittleOlTable[3]: ", anyLittleOlTable[3]) --prints '4445  better than ever'

    anyLittleOlTable[3] = "joojoojooba" --index 3  already in use so __newindex NOT called and value NOT adjusted
    print("anyLittleOlTable[3] again: ", anyLittleOlTable[3]) --prints only 'joojoojooba'

In this way we can interrupt the assignment of a value but still capture that value in the main table.

However, using rawset can have an unintended consequence. To understand let’s now make __index actually do something:

    usefulMeta.__index = function(tableBeingAccessed, key)
         print(“nope”) -- note that nothing is returned

Again, this will only be called when accessing usused indexes and keys:

     capturedValue = anyLittleOlTable[3] --index exists so value is passed normally and nothing else happens
     notCapturedValue = anyLittleOlTable[4] --index unused so “nope” is printed and nothing else happens (no value returned)

(by the way, what if we want to read from a table, and even if the value is nil, we don’t want __index to activate? That’s the time for rawget)

    notCapturedValue = anyLittleOlTable[4] --as above, “nope” is printed and nothing else happens (no value returned)
    notCapturedValue = rawget(anyLittleOlTable, 4) --still no value returned, but “nope” is NOT printed

So, okay, so far so good. We can make stuff happen when reading and writing to unused keys/indexes.

Still, only affecting unused keys/indexes is of limited value.

So let’s say we want both of our metamethods to always be called.

In other words, even if the indexes/keys are already used, we want “better than ever” to be added to every value ever put in the table. And we want every attempt to access values with keys or indexes to print out a teasing word.

This is where rawset causes trouble.

Because our current __newindex assigns values directly to indexes/keys of the main table, as soon as that happens those indexes/keys will no longer route to the metamethods. So as soon as the __newindex method defines a new key/index, it prevents itself from ever being called again by that key/index.

[[ unless that key/index gets nilled out at some point, in which case it becomes ‘unassigned’ again, and will once more route to the metamethod. ]]

So how do we fix this? The solution is to play a little shell game. It can be done by storing all the values in the metatable itself, and leaving the main table totally empty.

    usefulMeta.__newindex = function(tableBeingAccessed, key, newValue)
        local adjustedValue = tostring(newValue).." better than ever"
        usefulMeta[key] = adjustedValue --the metatable is now putting the key and value into itself

With this technique, since the main table is always empty, any attempts to read/write will always route to the metatable. And if then you do want to get at the stored values, you read them from the metatable.

So putting that all together:

     aDifferentMeta = {}
     aDifferentMeta.__newindex = function(tableBeingAccessed, key, newValue)
        local adjustedValue = tostring(newValue).." better than ever"
        usefulMeta[key] = adjustedValue --metatable is now putting the key and value into itself
     aDifferentMeta.__index = function(tableBeingAccessed, key)
         return aDifferentMeta[key] --leave this out to only print ‘yep’ and do nothing else

     aDifferentLittleOldTable = {}
     setmetatable(aDifferentLittleOldTable, aDifferentMeta)

    aDifferentLittleOldTable[1] = 8338007 --key/index is not used so __newindex adjusts value and stores it in aDifferentMeta
    print("aDifferentLittleOldTable[1]: ", aDifferentLittleOldTable[1]) --__index prints ‘yep’, then reads aDifferentMeta and prints '8338007 better than ever'

    aDifferentLittleOldTable[1] = "laffy taffy raffle" --key/index *still not used by main table*, so __newindex acts just like before
    print("aDifferentLittleOldTable[1] again: ", aDifferentLittleOldTable[1]) --__index prints ‘yep’ again, reads aDifferentMeta again, and prints 'laffy taffy raffle better than ever'


I hope that helps somebody!

….but just one more thing

In these examples each table has its very own metatable, but in practice different tables can also share a single metatable. In that case the metatable would have to make a new “shadow” table for each of the main tables, and read and write to that table:

     multiUseMeta = {}
     multiUseMeta.__newindex = function(tableBeingAccessed, key, newValue)

           if not tableBeingAccessed.utilityTable then
                 tableBeingAccessed.utilityTable = {} --making a ‘shadow’ table when first accessed

           local adjustedValue = tostring(newValue).." better than ever"
           tableBeingAccessed.utilityTable[key] = adjustedValue
     multiUseMeta.__index = function(tableBeingAccessed, key)
         if tableBeingAccessed.utilityTable then
                return tableBeingAccessed.utilityTable[key]

     tableSharingMeta = {}
     secondTableSharingMeta = {}
     setmetatable(tableSharingMeta, multiUseMeta)
     setmetatable(secondTableSharingMeta, multiUseMeta)

    tableSharingMeta[1] = “lucky dodger” -- __newindex adjusts value and stores it in tableSharingMeta.utilityTable
    secondTableSharingMeta[1] = "snake people" -- __newindex adjusts value and stores it in secondTableSharingMeta.utilityTable

    print("tableSharingMeta[1]: ", tableSharingMeta[1]) --__index prints ‘yep’, reads tableSharingMeta.utilityTable, prints 'lucky dodger better than ever'
    print("secondTableSharingMeta[1]: ", secondTableSharingMeta[1]) --__index prints ‘yep’, reads secondTableSharingMeta.utilityTable, prints 'snake people better than ever'

In this way, using a single metatable, we can add the same read/write behavior to every table we want.

…so, yeah, there you go!

Metatables can do a bunch of other stuff, but this is the read/write stuff in a nutshell. I think!

This is the explanation I wish I’d found online. So again, I hope this helps someone. If I got anything wrong, please let me know.

Hi @UberGoober … one thing … I didn’t think it’s true that having a meta table means you can’t access the base table. __index and __new index only intercept indexes that don’t already exist, unless I’m mistaken, which I could be.

I’ll read your thing in more detail when I get a chance. There are some metatable examples somewhere in my stuff … search for the word, maybe. But your examples look good at first glance!


@RonJeffries are you responding to the tl/dr?

I believe I stress repeatedly in the post that those methods only work on undefined keys.

Yes the text is good, and I think the tl;dr is a bit misleading.

I certainly see what you mean, but a tl/dr cannot by definition be complete.

I’m open to suggestions for revision, but I’d like to point out that one of the main frustrations I had with other explanations was that they put the fact of metamethods only functioning on unassigned keys front and center, which made them seem particularly useless for the purpose I had in mind, when the actual truth is that they can very easily be used for the purpose I had in mind.

So it is important to me that the summary explain that the metamethods can be fired at every single read and write, and it be left to the details to explain the intricacies involved.

I was mistaken. The tl;dr is fine. It’s this line that I think is misleading:

The main thing it took me a while to understand is that by default metatable methods prevent access to the main table.

No biggie either way. It’s a solid article.

Here’s a meta example I created when I was trying to understand them. Meta index and newindex are only called if a table key doesn’t exist yet. That’s either for reading a table entry or updating it. If the key exists then meta isn’t called. A table with meta methods can be used just like any other table except for whatever methods are assigned to it. I included the method __add here just for kicks. I guess you could include code there that would parse both tables and add up any entries that are numeric and return the sum. There are a lot more methods that can be included, but this was enough for me to see what was happening. I haven’t used meta methods in anything I coded except for examples, so I’m no expert on them.

function setup()
    print("key doesnt exist, calls meta index")
    a=tab1[3]   -- key doesnt exits, calls meta index
    print("key doesnt exist, calls meta newindex")
    tab1[3]=444 -- key doesnt exist, calls meta newinde

    print("key exists, no meta newindex call")
    tab1[3]=555 -- key exists, doesnt call meta update
    print("key exists, no meta index call")
    a=tab1[3]   -- key exits, doesnt call meta index
    print("any insert keys dont exist, calls meta newindex")
    table.insert(tab1,111)  -- table insert keys dont exist, calls meta newindex
    print("key 3 already exists, next is 4")
    print("trying to add 2 tables")
    a=tab1+tab1     -- trying to add 2 tables, calls meta add
    print("adding 2 numbers, no meta call")
    a=4+5           -- adding numbers, doesnt call meta add


function setUpMetaCalls()
    -- meta for reading a table, only if the key doesnt exist
        print("meta: "..key.." key not found")
    -- meta for updating a table, only if the key doesnt exist
        print("meta: updating new key "..key,"with "..val)
    -- meta for adding 2 tables
        print("meta: add some code here")

@RonJeffries the line is now “ The main thing it took me a while to understand is if a metamethod is activated, by default it prevents access to the main table.

Does that sit better?

I wouldn’t say it, but it’s your gift to the gang, not mine, let it stand.