Codea discussion title search

I wanted to play around with http.request, so I thought I would write a program that will allow a search of all the discussion titles in the Codea forum. It takes about a minute to read all 67 pages and extract the titles (50 per page), but once read you can scroll thru the titles (about 3,312 so far). There is a fast scroll (sliding your finger up/down the right side of the screen) or a slow scroll (sliding your finger up/down the left side of the screen). There is also a character by character search function that displays the titles containing what is keyed in the string area as each character is keyed. Just tap in the string area and key what you want to search for. There is a title count that shows how many titles are being shown, either all or just the matched titles. Use the backspace key to clear the string area to redisplay all the titles or to start a new search. When the program is started, the page currently being read is shown in the pageRequested area. Right now there are 67 pages, but that count is growing fast. If what you want to search for or look at isn’t very far back, a triple tap will abort the page request function and allow you to look at what is loaded so far. There isn’t an option to continue an aborted request, so you’ll have to restart the program if you want to load every page. To keep the program small, I didn’t add anything to save the titles or to add a function to display the text of any title discussion. To the left of each title is a page number so you can look in the forum on that page for the title you are interested in. You can flip between the Codea forum and this program to view any discussion. It’s possible that the page number won’t match the forum page if there were some discussions added since the program was started. I think this will be handy for the new coders or anyone else, because you can easily search the full forum for something you might be interested in or have a question about.


function setup()
    srch={}
    title={}
    searchFound=false
    pageDone=true
    doneReading=false
    page=0
    yOffset=0
    str1='http://www.twolivesleft.com/Codea/Talk/discussions'
    parameter.text("string","",searchTitle)
    parameter.watch("pageRequested")
    parameter.watch("titles")
end

function draw()
    background(40,40,50)
    fill(255)
    textMode(CORNER)
    getPages()
    if searchFound then
        showFound()
    else
        showAll()
    end
end 

function showAll()    -- show all titles
    line=0
    titles=0
    for z=1,#title do
        for y=1,#title[z] do
            titles=titles+1
            if titles>yOffset and titles<yOffset+26 then
                line=line+1
                text("--------------------------",1,HEIGHT-line*20)
                line=line+1
                text(z.." "..title[z][y],10,HEIGHT-line*20)
            end
        end
    end
end

function showFound()    -- show matched titles
    line=0
    titles=0
    for z=1,#srch do
        titles=titles+1
        if titles>yOffset and titles<yOffset+26 then
            line=line+1
            text("--------------------------",1,HEIGHT-line*20)
            line=line+1
            text(srch[z],10,HEIGHT-line*20)  
        end 
    end
end    

function getPages()    -- read next page
    if pageDone and not doneReading then
        page=page+1
        pageDone=false
        pageRequested=string.format("%s/p%d",str1,page)
        http.request(pageRequested,gotData,failed)
    end
end

function failed(error)    -- read error
    print(error)
end

function searchTitle()    -- search title for string
    yOffset=0
    srch={}
    for z=1,#title do
        for y=1,#title[z] do
            s,e=string.find(string.lower(title[z][y]),string.lower(string))
            if s~=nil then
                searchFound=true
                table.insert(srch,z.." "..title[z][y])
            end
        end
    end 
end

function gotData(data,status,headers)    -- page received
    title[page]={}
    findDiscussion(data,page)
end

function findDiscussion(str,pg)    -- find discussion titles by page
    s=1
    itemFound=false
    while true do
        s,e=string.find(str,'ItemContent Discussion',s)
        if s==nil then
            pageDone=true
            if not itemFound then
                pageRequested="complete"
                doneReading=true
            end
            return
        end
        itemFound=true
        s1,e1=string.find(str,'"Title">',e)
        s2,e2=string.find(str,'</a>',e1)
        st=string.sub(str,e1+1,s2-1)
        table.insert(title[pg],st)
        s=e2
    end
end

function touched(t)
    if t.state==BEGAN and t.tapCount==3 then    -- abort page request
        doneReading=true
        pageRequested="request aborted"
    end
    if t.state==MOVING then
        if t.x>WIDTH/2 then    -- right side of screen
            yOffset=yOffset+t.deltaY    -- fast scroll
        else
            yOffset=yOffset+t.deltaY/40    -- slow scroll
        end
        if yOffset<0 then    -- top of list
            yOffset=0
        end
        if yOffset+10>titles then -- bottom of list
            yOffset=titles-10
        end
    end
end 

Wow! Great! =D>

Nice. It would be great if you could press on an item and have that page come up in the in app web browser…

@Dave1707-try the code below

if you press on the number of any item, it will load in Codea’s internal web browser. You can toggle a parameter to use Safari instead

This is done by extracting the URL of each item at the same time as finding its title. It should work with search as well.

Note I had to rewrite the table of titles to just be a 1D list, ie not store titles by page (because the discussion page doesn’t matter once you can bring the actual discussion up).

All edits are marked with --NEW


function setup()
    srch={}
    title={}
    discuss={}   --NEW holds URLs
    searchFound=false
    pageDone=true
    doneReading=false
    page=0
    yOffset=0
    str1='http://www.twolivesleft.com/Codea/Talk/discussions'
    parameter.text("string","",searchTitle)
    parameter.watch("pageRequested")
    parameter.watch("titles")
    parameter.boolean("Open page in Safari")   --NEW 
end

function draw()
    background(40,40,50)
    fill(255)
    textMode(CORNER)
    getPages()
    if searchFound then
        showFound()
    else
        showAll()
    end
end 

function showAll()    -- show all titles
    line=0
    titles=0
    for z=1,#title do
        titles=titles+1
        if titles>yOffset and titles<yOffset+26 then
            line=line+1
            text("--------------------------",1,HEIGHT-line*20)
            line=line+1
            text(z.." "..title[z],10,HEIGHT-line*20)
        end
    end
end

function showFound()    -- show matched titles
    line=0
    titles=0
    for z=1,#srch do
        titles=titles+1
        if titles>yOffset and titles<yOffset+26 then
            line=line+1
            text("--------------------------",1,HEIGHT-line*20)
            line=line+1
            text(srch[z],10,HEIGHT-line*20)  
        end 
    end
end    

function getPages()    -- read next page
    if pageDone and not doneReading then
        page=page+1
        pageDone=false
        pageRequested=string.format("%s/p%d",str1,page)
        http.request(pageRequested,gotData,failed)
    end
end

function failed(error)    -- read error
    print(error)
end

function searchTitle()    -- search title for string
    yOffset=0
    srch={}
    srchDiscuss={}   --NEW 
    for z=1,#title do
            s,e=string.find(string.lower(title[z]),string.lower(string))
            if s~=nil then
                searchFound=true
                table.insert(srch,z.." "..title[z])
                table.insert(srchDiscuss,discuss[z])   --NEW
            end
    end 
end

function gotData(data,status,headers)    -- page received
    findDiscussion(data)
end

function findDiscussion(str)    -- find discussion titles by page
    s=1
    itemFound=false
    while true do
        s,e=string.find(str,'ItemContent Discussion',s)
        if s==nil then
            pageDone=true
            if not itemFound then
                pageRequested="complete"
                doneReading=true
            end
            return
        end
        itemFound=true
        s1,e1=string.find(str,'/Codea/Talk/discussion/',e)   --NEW 
        s2,e2=string.find(str,'"',e1)   --NEW 
        st=string.sub(str,e1+1,s2-1)   --NEW 
        table.insert(discuss,st)   --NEW 
        s1,e1=string.find(str,'"Title">',e)
        s2,e2=string.find(str,'</a>',e1)
        st=string.sub(str,e1+1,s2-1)
        table.insert(title,st)
        s=e2
    end
end

function touched(t)
    if t.state==BEGAN and t.tapCount==3 then    -- abort page request
        doneReading=true
        pageRequested="request aborted"
    end
    if t.state==MOVING then
        if t.x>WIDTH/2 then    -- right side of screen
            yOffset=yOffset+t.deltaY    -- fast scroll
        else
            yOffset=yOffset+t.deltaY/40    -- slow scroll
        end
        if yOffset<0 then    -- top of list
            yOffset=0
        end
        if yOffset+10>titles then -- bottom of list
            yOffset=titles-10
        end
    end
       --NEW all new below
    if t.state==ENDED and t.x<WIDTH/4 then
        local n=math.floor((768-t.y)/40+1+yOffset) --get discussion number
        local browser=true
        if Open_page_in_Safari then browser=false end
        local tbl=discuss
        if #srchDiscuss>0 then tbl=srchDiscuss end
        openURL('http://www.twolivesleft.com/Codea/Talk/discussion/'..tbl[n],browser)
    end
end 

@Ignatz Thanks for the update. In my other version of this code, I have a discussion table where I already saved the discussion URL for each title. I just didn’t know what to do with it so I took it out of this posted version. I’ll see how you did the safari call and get the discussion table back in. The only problem is I’m off to bed and in the morning I have other things to do so I won’t be working on this for about 14 hours.

@Ignatz, @dave1707, great!

It adds great value to this forum! After you become a “codea expert”, you want to share your acquired knowledge with other people, and this little codea tool make it easy: it’s just a matter of looking for interesting questions, which you gladly would like to answer (even the prehistoric ones). >:D<

Here is the latest update of my code with the openURL command thanks to @Ignatz. I removed the line numbers that displayed with the titles. I added a topLine display to show you what title number is at the top of the screen. That’s so you know where you are in the list of titles. To display the discussion of a title, you need to double tap that title. Once in the discussion, tapping “done” in the upper left corner will return you back to this program. I used the double tap so it wouldn’t interfere with the scrolling of the titles. A triple tap still aborts the page request. @Ignatz When I ran your program, I always got the wrong discussion and when I tried the discussion of a searched title, I would get an error on the openURL line. None of that really mattered because you provided the openURL command that I needed. Thanks again. There is one problem that I noticed with this code. Sometimes when returning to the program from the discussion page, the title area would be blank and the program seems to be running, but things don’t work right. If I try doing a search, I get an error message saying something like “the called function is nil”. If you run into any problems let me know so I can update the program.


function setup()
    title={}
    discussion={}
    srchTitle={}
    srchDisc={}
    searchFound=false
    pageDone=true
    doneReading=false
    page=0
    yOffset=0
    titles=0
    topLine=0
    str1='http://www.twolivesleft.com/Codea/Talk/discussions'
    str2='http://www.twolivesleft.com'
    parameter.text("string","",searchTitle)
    parameter.watch("pageRequested")
    parameter.watch("topLine")
    parameter.watch("titles")
end

function draw()
    background(40,40,50)
    fill(255)
    textMode(CORNER)
    getPages()
    if searchFound then
        showFound()
    else
        showAll()
    end
end 

function showAll()    -- show all titles
    line=0
    titles=0
    for z=1,#title do
        titles=titles+1
        if z>yOffset and z<yOffset+26 then
            if topLine==0 then
                topLine=z
            end
            line=line+1
            text("--------------------------",1,HEIGHT-line*30)
            line=line+1
            text(title[z],10,HEIGHT-line*30)
        end
    end
end

function showFound()    -- show matched titles
    line=0
    titles=0
    for z=1,#srchTitle do
        titles=titles+1
        if z>yOffset and z<yOffset+26 then
            if topLine==0 then
                topLine=z
            end
            line=line+1
            text("--------------------------",1,HEIGHT-line*30)
            line=line+1
            text(srchTitle[z],10,HEIGHT-line*30)  
        end 
    end
end    

function getPages()    -- read next page
    if pageDone and not doneReading then
        page=page+1
        pageDone=false
        pageRequested=string.format("%s/p%d",str1,page)
        http.request(pageRequested,gotData,failed)
    end
end

function failed(error)    -- read error
    print(error)
end

function searchTitle()    -- search title for string
    yOffset=0
    topLine=0
    srchTitle={}
    srchDisc={}
    for z=1,#title do
        s,e=string.find(string.lower(title[z]),string.lower(string))
        if s~=nil then
            searchFound=true
            table.insert(srchTitle,title[z])
            table.insert(srchDisc,discussion[z])
        end
    end 
end

function gotData(data,status,headers)    -- page received
    findDiscussion(data,page)
end

function findDiscussion(str,pg)    -- find discussion titles by page
    s=1
    itemFound=false
    while true do
        s,e=string.find(str,'ItemContent Discussion',s)
        if s==nil then
            pageDone=true
            if not itemFound then
                pageRequested="complete"
                doneReading=true
            end
            return
        end
        itemFound=true
        s1,e1=string.find(str,'href="',e)
        s2,e2=string.find(str,'"',e1+1)
        st=string.sub(str,e1+1,s2-1)
        table.insert(discussion,st)        
        s1,e1=string.find(str,'"Title">',e2)
        s2,e2=string.find(str,'</a>',e1)
        st=string.sub(str,e1+1,s2-1)
        table.insert(title,st)
        s=e2
    end
end

function touched(t)
    if t.state==BEGAN and not donereading and t.tapCount==3 then
        doneReading=true
        pageRequested="request aborted"    
    end
    if t.state==MOVING then
        topLine=0
        if t.x>WIDTH/2 then    -- right side of screen
            yOffset=yOffset+t.deltaY    -- fast scroll
        else
            yOffset=yOffset+t.deltaY/40   -- slow scroll
        end
        if yOffset<0 then    -- top of list
            yOffset=0
        end
        if yOffset+6>titles then -- bottom of list
            yOffset=titles-6
        end
    end
    if t.state==BEGAN and doneReading and t.tapCount==2 then
        showKeyboard()
        hideKeyboard()
        tap=math.ceil((HEIGHT-t.y)/60+math.floor(yOffset))
        if searchFound then
            if tap<=#srchDisc then
                openURL('http://www.twolivesleft.com'..srchDisc[tap],true)
            end
        else
            if tap<=#discussion then
                openURL('http://www.twolivesleft.com'..discussion[tap],true)
            end
        end
    end
end 

@dave1707 - funny, it worked for me

If you want to make this a serious utility, it should ideally avoid having to read all the same stuff every time you run it. That means storing the data locally and adding new discussions to it, when it starts. Then it would be ready almost instantly.

(However, to be honest, I think I prefer a normal google search because that looks at the discussion text too, and I don’t have to leave the Codea project I’m currently working on).

@Ignatz I’ll have to load your code again and try it. It was late when I loaded it and I was in a hurry to get to bed. Had a lot of stuff to do in the morning. Maybe I messed something up in my haste. I was thinking about saving the titles, but when I found out there was over 3,000 , I didn’t want to waste the space. I didn’t think anyone else would want to either. As for making this a serious utility, not me. I’m retired, nothing I do anymore is serious. I had 45 years of serious work (35 years coding) and that was enough. I originally started this as an easy way for the new coders to look/search thru all of the titles. Then I was thinking of searching thru the discussions, but that would have involved too much reading so I dropped that. Since I haven’t messed around with network stuff that much, I didn’t know/remember about the openURL command. That made things easy to get the discussion text. Thanks.

@dave1707 - wait until you see a new code sharing project currently under development. It does some amazing things with openURL (not my work, I hasten to add!).

(PS I’m also retired, and I also don’t want to slog my way through anything too serious!)