Possibly better (for some cases) remove(...) function

The problem with table.remove that I see is that we have to keep track of what is in what position.

For my usage I want an array to loop through numerically, and then dynamically remove an item from the list, while everything else shifts down, still able to keep track of what is at which index, so I can remove or add anything else at any moment.

@UberGoober ‘s solution would actually work well for me because my values are functions, and each function is unique even if it’s structurally identical, so I wouldn’t need to worry about breaking out of the loop too early. A better question is when would you as a programmer be unsure if your table is an array or a hashmap? It should always be pretty obvious which one you’re working with

side note - @UberGoober the share button for each answer is at the bottom of the answer on the left hand side opposite the answerer’s name

@skar that’s the one. How did you link directly to that answer and not the whole question?

@UberGoober When you use a “for” loop to iterate thru a table and you use table.remove(), when a matched item is removed, all the items above it are moved down 1 position to fill the void. The “for” index is increased by 1 and the next item is checked. The problem is table.remove() doesn’t check the item that was moved into the empty position, it’s skipped, because the “for” loop moved on to the next position. To eliminate that problem, you iterate thru the table in reverse order, from the end to the beginning. So it doesn’t matter if the items are moved down to fill the position of the deleted item, you’ve already checked all of them. And because you’re going down the table, nothing below the position you’re at have moved.

if i recall the lua spec, it does not say when a table will be in sequence form, and when not.

The length operator applied on a table returns a border in that table. A border in a table t is any natural number that satisfies the following condition:

(border == 0 or t[border] ~= nil) and t[border + 1] == nil

In words, a border is any (natural) index present in the table that is followed by an absent index (or zero, when index 1 is absent).

it goes on to say:

When t is a sequence, #t returns its only border, which corresponds to the intuitive notion of the length of the sequence. When t is not a sequence, #t can return any of its borders. (The exact one depends on details of the internal representation of the table, which in turn can depend on how the table was populated and the memory addresses of its non-numeric keys.)

I’m not sure but this seems to say the opposite of what you’re suggesting it says.

It reads to me that if the table is “a sequence” then #table is consistently the length of that sequence, and if it’s not, it’s unpredictable.

Values with sequential numerical keys behave like a standard array and their length can be measured reliably with #, and if not, not.

An array with a nil in the middle is not a sequence. The rule is saying that if there is a hole in an array, Lua specifically says that # can return any bound, which may or may not be the last one. In short, if you nil an element of an array, # is no longer guaranteed to tell us what we think it should.

Here’s a test:

-- RJ 20200911
-- Delete these and replace with your own.

function testArraysWithNils()
    CodeaUnit.detailed = true

    _:describe("# may not be doing what you expect", function()

        _:before(function()
            ary = {}
            for i = 1,100 do
                table.insert(ary, i)
            end
        end)

        _:after(function()
        end)

        _:test("# works", function()
            _:expect(#ary).is(100)
            ary[50] = nil
            _:expect(#ary).is(99) -- fails with 100
            local count = 0
            for i,x in ipairs(ary) do
                count = count + 1
            end
            _:expect(count).is(99) -- fails with 49!!
            count = 0
            for i,x in pairs(ary) do
                count = count + 1
            end
            _:expect(count).is(99) -- passes
            local deletes = {14, 54, 67, 34, 19, 81, 55, 64, 75, 93}
            for i,x in ipairs(deletes) do
                ary[x] = nil
            end
            _:expect(#ary).is(100- #deletes - 1) -- fails with 100
            ary[135]=135
            _:expect(#ary).is(100 - #deletes) -- fails with 63!!
        end)

    end)
end

The last one is interesting. Before that assignment of 135, there are 89 non-nil elements in the array, the initial 100, minus the one at 50, minus the ten from the deletes loop. After the assignment into 135, there are 89 non-nils. # reports 63.

You can’t trust # if there are nils in the table.

@RonJeffries I don’t know where you think you and I are saying different things. You seem to be arguing against something I don’t think I asserted. I’m sure I could be wrong in either this assertion or that one, but if so, can we just say you aren’t telling me anything I intended to disagree with?

@RonJeffries also tests don’t really say anything about my function’s validity when they aren’t testing my function, no?

i’m not arguing against anyone or anything. i’m pointing out a fact, namely that # works in a literally undefined fashion when there are nils left in an array. i’ve not tried testing your function. if i wanted to remove things from a table, i’d most likely use a hash-style, since then remove is O(1), i.e. very fast.

your function, as revised, may be valid. i don’t know, and don’t have an opinion. would you like me to study it and/or test it?

@RonJeffries, I understand the point you’re making, I just don’t know who you’re thinking needs to hear it.

I’d be honored if you wanted to poke holes in my code, but I know you’re a busy man and I’d hate to delay your Dung work. :slight_smile:

Honesty, I’m totally okay if you have no interest.

Looking at this:

  if shouldMoveDown then
                --check if we're not at the end
                if i ~= #targetTable then
                    --if not, copy the next value over this one
                    targetTable[i] = targetTable[i+1]
                else
                    --if so, delete the last value
                    targetTable[#targetTable] = nil
                end 
            end

it seems to me that the if/else isn’t needed. in the else, i is #targetTable, and therefore targetTable[i+1] is nil.

i have not tested that hypothesis.

I think the point is not to make i + 1 nil but to make the last element of the ordered table nil, since everything else has been moved down one index.

point is, since i+1 is past the end, it will be nil, setting i == last to nil. i’ve not tested it but i think the i = i+1 statement works at the end w/o the special check. i could be wrong, but don’t think i am.

You’re right I’m wrong you’re big I’m small you’re smart I’m dumb.
—paraphrased from “Matilda”

I commented out the “then” part and it still passes all my tests.

Nicely spotted.

Nice to have those tests!