string.gmatch eating framerate

Questions
1

Alright so im making a little program that can read xml files, which i use string.gmatch to filter out all the information i need. But when i run huge files trough the loop, my framerate drops all the way to 0
I can’t really show any functional code, since its for something i will get payed for.
I just wonder if theres any way to make it not eat my framerate and/or if this is a known issue.

Example of the loop im using:

for k,b in string.gmatch(SomeStringHere, SomePattern) do 
   -- do stuff here
end

EDIT: got it fixed, used a different way which is way quicker

Still dont know what caused the problem though

2

  1. How many gmatches can you do before t gets slow?

  2. Do you need to do all the gmatches in one frame, or can you spread them across several frames, eg if you only want a 3 second response time, you can spread your searches across 180 frames.

3

How come you need to read an xml file every frame? Why not just read it once, parse the information into something that’s very quick to access, like a table, and access that every frame?

4

@Ignatz @yojimbo2000 No, no. My bad. Im not calling it every frame, just once at the start.

5

How impatient are the users? Either put up a “please wait a moment” sign, or place the gmatch in a coroutine to give the user a gradually updating status bar (though that will almost certainly slow it down even further, eg if yield is inside the gmatch loop then each iteration would take at least a frame, so I wouldn’t recommend that). Can you at least show us the pattern in the gmatch? How many iterations are you getting from these huge files?

6

@Luxizzle How big are the files you’re using. I have a 3,151,541 character dictionary file (300249 words) that I use. If I do a string.gmatch on the last word, it reads the whole file in the blink of an eye. What are you doing with the results of gmatch, maybe that’s what’s slowing things down.

7

Anchoring the search with ^ and $ helps a lot.

8

@Luxizzle Since you don’t want to show any code or say what you’re doing, I’ll take a guess that you want to view selected information from an xml file. Here’s an example I have that reads an html file, but xml is probably the same. I have this example setup to show any text between the < and > characters in the file. You can drag your finger up or down the screen to see all the lines. Let me know if this is usable or not close to what you want.

displayMode(FULLSCREEN)
supportedOrientations(LANDSCAPE_ANY)

function setup()
    font("Courier")
    textMode(CORNER)
    st=1
    str=""
    tab={}
    dy=0
    url="https://github.com/cbracco/html5-test-page/blob/master/test.html"    
    http.request(url,response)
    msg="Please wait, reading file"
end

function response(data,status,headers)
    for a,b,c in string.gmatch(data,"(<)(.-)(>)") do
        table.insert(tab,b)
    end
end

function draw()
    background(40, 40, 50)
    fill(255)
    if #tab==0 then
        text(msg,350,350)
    else
        st=math.ceil(dy/20)+1
        en=math.min(st+50,#tab)
        line=0
        for z=st,en do
            line=line+1
            str=string.format("%4d %s",z,string.sub(tab[z],1,90))
            text(str,0,HEIGHT-line*25)
        end
    end
end

function touched(t)
    if t.state==MOVING then
        dy=math.max(dy+t.deltaY,0)
    end
end