|
Post by alexfish on Jul 2, 2017 15:53:58 GMT 1
Hi All
A simple search engine based on CURL & DuckDuckGo
I have not work out some of the codes in the url links. yet most are readable will update as on going if can work out some of the codes..
Have Fun
Alex
' Simple Terminal Search Engine > requires Curl LOCAL x LOCAL Search$= "curl https://duckduckgo.com/html/?q="
INPUT "Search DuckDuckGo ......> " ,Inp$ ' Must hav a result ------------------------------ IF LEN(Inp$) > 0 THEN
ELSE Inp$ = "News"
END IF
PRINT "Searching for:: " & Inp$
SPLIT Inp$ BY " " TO Bits$ SIZE x
FOR n = 0 TO x-1
IF LEN(Bits$[n])>0 THEN Bits$[n]= CHOP$ (Bits$[n])
IF n < x-1 THEN Search$=Search$ & Bits$[n] & "%" ELSE Search$=Search$ & Bits$[n] ENDIF
ENDIF
NEXT
HTML$ = EXEC$(Search$) IF NOT(INSTR(HTML$,"<a")) THEN PRINT " Can not Resolve Host : DuckDuckGo" PRINT " Check Internet Connection" END END IF 'Clean Up and save to file
SPLIT HTML$ BY "\n" TO Res$ SIZE x
FOR n = 0 TO x-1
Res$[n]= CHOP$ (Res$[n]) IF LEN(Res$[n])>0 THEN tem$ = LEFT$(Res$[n],1) IF tem$ == "<" THEN Results$ = CHOP$(Results$ & Res$[n]) & "\n" END IF END IF
NEXT
SAVE Results$ TO CURDIR$ & "/index.html" IF FILEEXISTS(CURDIR$&"/index.html") THEN
PRINT "Results saved in --> " & CURDIR$&"/index.html" Result$ = ""
OPEN CURDIR$&"/index.html" FOR READING AS myfile WHILE NOT(ENDFILE(myfile)) DO READLN txt$ FROM myfile txt$ = CHOP$(txt$) IF NOT(ENDFILE(myfile)) THEN IF LEFT$(txt$,2) == "<a" THEN
txt$ = REPLACE$(txt$,"'","'") txt$ = REPLACE$(txt$,"</b>","") txt$ = REPLACE$(txt$,"<b>","") txt$ = REPLACE$(txt$,"</a>","") txt$ = REPLACE$(txt$,"%3A",":") txt$ = REPLACE$(txt$,"%2F","/") txt$ = REPLACE$(txt$,"%2D","-") IF INSTR(txt$,"nofollow") THEN HTMLTAG$ = LEFT$(txt$,INSTR(txt$,">")) END IF
IF (INSTR(txt$,"nofollow" )) THEN COLOR FG TO YELLOW ELSE COLOR RESET END IF
PRINT MID$(txt$,INSTR(txt$,">")+1) IF INSTR(txt$,"result__url") THEN COLOR FG TO RED HTMLTAG$ =MID$(HTMLTAG$,INSTR(HTMLTAG$,"http")) HTMLTAG$= LEFT$(HTMLTAG$,LEN(HTMLTAG$)-2) PRINT HTMLTAG$ COLOR FG TO BLUE PRINT PRINT "**************************************************" PRINT COLOR RESET
END IF Result$ = Result$ & txt$
END IF ENDIF WEND CLOSE FILE myfile
END IF
|
|
|
Post by alexfish on Jul 2, 2017 19:34:30 GMT 1
Hi all further enhancements & filters this on can link to a default Browser if xdg-open works on start 'enter:: y ' to enable the xdg-open after a search option to continue y/n If call to browser enter 0 to pass or 1 TO If wondering why INPUT STRING , I be getting a Seg fault , yet testing of the doc code INPUT "What is your age? ", age PRINT "You probably were born in ", YEAR(NOW) - age Works . so need to debug that one BR Alex Found a BUG looks like a callback to search again = something odd with the links count ? have updated code to reset the count to 0 ,but still the same After a call to the browser m can exit for now ' Simple Terminal Search engine > requires Curl ' this also requires default command "xdg-open"
PROTO system LOCAL links TYPE int
LOCAL x=0 LOCAL browser=0 LOCAL tags=0 LOCAL Search$= "curl https://duckduckgo.com/html/?q=" LOCAL ALLHTMLTAGS$ INPUT "See result in default browser y/n> " ,Inp$ Inp$=UCASE$(Inp$) IF INSTR(Inp$,"Y")==1 THEN browser=1 END IF
LABEL start tags=0 ALLHTMLTAGS$="" INPUT "Search DuckDuckGo ......> " ,Inp$ CLEAR IF Inp$ == "quit" THEN
INPUT " Quit y/n > " ,test$ test$= CHOP$(test$)
IF (NOT(LEN(test$))) THEN END ELSE
test$ = UCASE$(test$) IF INSTR(test$,"Y") THEN
END ENDIF END IF END IF ' Must hav a result ------------------------------ IF LEN(Inp$) > 0 THEN
ELSE Inp$ = "News"
END IF
PRINT "Searching for:: " & Inp$
SPLIT Inp$ BY " " TO Bits$ SIZE x
FOR n = 0 TO x-1
IF LEN(Bits$[n])>0 THEN Bits$[n]= CHOP$ (Bits$[n])
IF n < x-1 THEN Search$=Search$ & Bits$[n] & "%" ELSE Search$=Search$ & Bits$[n] ENDIF
ENDIF
NEXT
HTML$ = EXEC$(Search$) IF NOT(INSTR(HTML$,"<a")) THEN PRINT " Can not Resolve Host : DuckDuckGo" PRINT " Check Internet Connection" END END IF 'Clean Up and save to file
SPLIT HTML$ BY "\n" TO Res$ SIZE x
FOR n = 0 TO x-1
Res$[n]= CHOP$ (Res$[n]) IF LEN(Res$[n])>0 THEN tem$ = LEFT$(Res$[n],1) IF tem$ == "<" THEN Results$ = CHOP$(Results$ & Res$[n]) & "\n" END IF END IF
NEXT
SAVE Results$ TO CURDIR$ & "/index.html" IF FILEEXISTS(CURDIR$&"/index.html") THEN
PRINT "Results saved in --> " & CURDIR$&"/index.html" Result$ = ""
OPEN CURDIR$&"/index.html" FOR READING AS myfile WHILE NOT(ENDFILE(myfile)) DO READLN txt$ FROM myfile txt$ = CHOP$(txt$) IF NOT(ENDFILE(myfile)) THEN IF LEFT$(txt$,2) == "<a" THEN 'http://baconbashtexas.com/%3Fproduct%3D45 'baconbashtexas.com/?product=45
txt$ = REPLACE$(txt$,"'","'") txt$ = REPLACE$(txt$,"</b>","") txt$ = REPLACE$(txt$,"<b>","") txt$ = REPLACE$(txt$,"</a>","") txt$ = REPLACE$(txt$,"%3A",":") txt$ = REPLACE$(txt$,"%2F","/") txt$ = REPLACE$(txt$,"%2D","-") txt$ = REPLACE$(txt$,"%3D","=") txt$ = REPLACE$(txt$,"%3F","?") txt$ = REPLACE$(txt$,"%23","#") IF INSTR(txt$,"nofollow") THEN HTMLTAG$ = LEFT$(txt$,INSTR(txt$,">")) END IF
IF (INSTR(txt$,"nofollow" )) THEN COLOR FG TO YELLOW INCR tags IF ODD(tags) THEN PRINT STR$(tags) & " ..." ELSE COLOR FG TO RED PRINT PRINT STR$(tags) & " _" END IF ELSE COLOR RESET END IF
PRINT MID$(txt$,INSTR(txt$,">")+1) IF INSTR(txt$,"result__url") THEN COLOR FG TO RED HTMLTAG$ =MID$(HTMLTAG$,INSTR(HTMLTAG$,"http")) HTMLTAG$= LEFT$(HTMLTAG$,LEN(HTMLTAG$)-2) PRINT HTMLTAG$ ALLHTMLTAGS$=ALLHTMLTAGS$ & HTMLTAG$ & "\n" COLOR FG TO BLUE PRINT PRINT "**************************************************" PRINT COLOR RESET
END IF Result$ = Result$ & txt$
END IF ENDIF WEND CLOSE FILE myfile END IF tags = (tags -1)/2 IF browser THEN PRINT " Select 1 TO " ,tags , " or " , 0 , " continue" INPUT "Select Link number Open Browser > " ,links$ links=VAL(links$) PRINT " Value ", links IF links== 0 THEN
GOTO contin END IF IF ODD(links) THEN links = links +1 END IF PRINT links SPLIT ALLHTMLTAGS$ BY "\n" TO HTP$ SIZE t PRINT " total > " , t-1 IF (links/2<=t-1) THEN PRINT " SELECTED > " ,links/2 PRINT HTP$[(links/2)-1] xdg$ = "xdg-open " & HTP$[(links/2)-1] & " &" system(xdg$) ELSE
GOTO contin 'PRINT HTP$[links] END IF END IF
LABEL contin INPUT "Continue y/n > " ,test$ test$= CHOP$(test$)
IF (NOT(LEN(test$))) THEN END ELSE
test$ = UCASE$(test$) IF INSTR(test$,"Y") THEN
GOTO start ENDIF END IF
|
|
|
Post by vovchik on Jul 2, 2017 20:13:11 GMT 1
Dear Alex,
Thanks. What about using "curl.bac"? It should be possible, too. I have been playing with it and can grab the bacon site but am getting errors (due to my ignorance) when I try https.
With kind regards, vovchik
|
|
|
Post by alexfish on Jul 2, 2017 21:31:08 GMT 1
Hi Vovchik Yes , that is the target If can help in that area then since I have Just found what is happening with this demo. Looks like the link to DuckDuckGo gets cached , in other words adds more and more to the fist search list . not sure what the actual limit is for now have corrected the counter. will add pages to this demo later. also need to find a reset if one changes the search this it where Demo is now looking ahead if can get this each link through rsvg then pass the image to a widget with callback .. and so on Added or pass it throw html2text & cross reference the bits BR Alex ' Simple Terminal Search engine > requires Curl ' this also requires default command "xdg-open" LOCAL colselect PROTO system LOCAL links TYPE int LOCAL lapptr =0 LOCAL x=0 LOCAL browser=0 LOCAL tags=0 LOCAL Search$= "curl https://duckduckgo.com/html/?q=" LOCAL ALLHTMLTAGS$ INPUT "See result in default browser y/n> " ,Inp$ Inp$=UCASE$(Inp$) IF INSTR(Inp$,"Y")==1 THEN browser=1 END IF
LABEL start COLOR RESET LOCAL Search$= "curl https://duckduckgo.com/html/?q=" tags=0 links=0 ALLHTMLTAGS$="" CLEAR INPUT "Search DuckDuckGo ......> " ,Inp$
IF Inp$ == "quit" THEN
INPUT " Quit y/n > " ,test$ test$= CHOP$(test$)
IF (NOT(LEN(test$))) THEN END ELSE
test$ = UCASE$(test$) IF INSTR(test$,"Y") THEN
END ENDIF END IF END IF ' Must hav a result ------------------------------ IF LEN(Inp$) > 0 THEN
ELSE Inp$ = "News"
END IF
PRINT "Searching for:: " & Inp$
SPLIT Inp$ BY " " TO Bits$ SIZE x
FOR n = 0 TO x-1
IF LEN(Bits$[n])>0 THEN Bits$[n]= CHOP$ (Bits$[n])
IF n < x-1 THEN Search$=Search$ & Bits$[n] & "%" ELSE Search$=Search$ & Bits$[n] ENDIF
ENDIF
NEXT
HTML$ = EXEC$(Search$) IF NOT(INSTR(HTML$,"<a")) THEN PRINT " Can not Resolve Host : DuckDuckGo" PRINT " Check Internet Connection" END END IF 'Clean Up and save to file
SPLIT HTML$ BY "\n" TO Res$ SIZE x
FOR n = 0 TO x-1
Res$[n]= CHOP$ (Res$[n]) IF LEN(Res$[n])>0 THEN tem$ = LEFT$(Res$[n],1) IF tem$ == "<" THEN Results$ = CHOP$(Results$ & Res$[n]) & "\n" END IF END IF
NEXT
SAVE Results$ TO CURDIR$ & "/index.html" IF FILEEXISTS(CURDIR$&"/index.html") THEN
PRINT "Results saved in --> " & CURDIR$&"/index.html" Result$ = ""
OPEN CURDIR$&"/index.html" FOR READING AS myfile WHILE NOT(ENDFILE(myfile)) DO READLN txt$ FROM myfile txt$ = CHOP$(txt$) IF NOT(ENDFILE(myfile)) THEN IF LEFT$(txt$,2) == "<a" THEN IF (INSTR (txt$,"Feedback")) AND (lapptr>0) THEN DECR tags PRINT " LOOOP +++++++++++++++++++++++++++++++++++++++++++" END IF 'http://baconbashtexas.com/%3Fproduct%3D45 'baconbashtexas.com/?product=45
txt$ = REPLACE$(txt$,"'","'") txt$ = REPLACE$(txt$,"</b>","") txt$ = REPLACE$(txt$,"<b>","") txt$ = REPLACE$(txt$,"</a>","") txt$ = REPLACE$(txt$,"%3A",":") txt$ = REPLACE$(txt$,"%2F","/") txt$ = REPLACE$(txt$,"%2D","-") txt$ = REPLACE$(txt$,"%3D","=") txt$ = REPLACE$(txt$,"%3F","?") txt$ = REPLACE$(txt$,"%23","#") IF INSTR(txt$,"nofollow") THEN HTMLTAG$ = LEFT$(txt$,INSTR(txt$,">")) END IF
IF (INSTR(txt$,"nofollow" )) THEN
INCR tags IF ODD(tags) THEN COLOR FG TO YELLOW PRINT STR$(tags) & " ..." COLOR RESET ELSE COLOR FG TO RED PRINT PRINT STR$(tags) & " _" COLOR RESET END IF ELSE COLOR RESET END IF
PRINT MID$(txt$,INSTR(txt$,">")+1) IF INSTR(txt$,"result__url") THEN COLOR FG TO RED HTMLTAG$ =MID$(HTMLTAG$,INSTR(HTMLTAG$,"http")) HTMLTAG$= LEFT$(HTMLTAG$,LEN(HTMLTAG$)-2) PRINT HTMLTAG$ ALLHTMLTAGS$=ALLHTMLTAGS$ & HTMLTAG$ & "\n" COLOR FG TO BLUE PRINT PRINT "**************************************************" PRINT COLOR RESET
END IF Result$ = Result$ & txt$
END IF ENDIF WEND CLOSE FILE myfile END IF tags = (tags -1)/2 IF browser THEN PRINT " Select 1 TO " ,tags , " or " , 0 , " continue" INPUT "Select Link number Open Browser > " ,links$ links=VAL(links$) PRINT " Value ", links IF links== 0 THEN
GOTO contin END IF IF ODD(links) THEN links = links +1 END IF PRINT links SPLIT ALLHTMLTAGS$ BY "\n" TO HTP$ SIZE t PRINT " total > " , t-1 IF (links/2<=t-1) THEN PRINT " SELECTED > " ,links/2 PRINT HTP$[(links/2)-1] 'xdg$ = "xdg-open " & HTP$[(links/2)-1] & " &" 'system(xdg$) 'xdg$="" ELSE
GOTO contin 'PRINT HTP$[links] END IF END IF
LABEL contin INPUT "Continue y/n > " ,test$ test$= CHOP$(test$)
IF (NOT(LEN(test$))) THEN END ELSE
test$ = UCASE$(test$) IF INSTR(test$,"Y") THEN INCR lapptr GOTO start ENDIF END IF
|
|
|
Post by alexfish on Jul 3, 2017 2:12:22 GMT 1
Hi All
This is a base line
need to compile this as
baconduckduck
then place in the /usr/bin
/usr/bin/baconduckduck
the code::
' program "baconduckduck " A Simple Terminal Search engine > requires Curl ' this needs to be in /usr/bin ' sudo cp baconduckduck /usr/bin/baconduckduck PROTO system LOCAL Inp$ LOCAL tags=0 LOCAL links TYPE int LOCAL lapptr =0 LOCAL browser=0
IF argc <2 THEN
Inp$="News" ELSE
FOR y = 1 TO argc-1 Inp$=Inp$ & argv[y] & "%" NEXT END IF
LOCAL Search$= "curl https://duckduckgo.com/html/?q=" & Inp$
ALLHTMLTAGS$=""
' Must hav a result ------------------------------ IF LEN(Inp$) > 0 THEN
ELSE Inp$ = "News"
END IF
PRINT "DuckDuckGo Searching for:: " & Inp$
SPLIT Inp$ BY " " TO Bits$ SIZE x
FOR n = 0 TO x-1
IF LEN(Bits$[n])>0 THEN Bits$[n]= CHOP$ (Bits$[n])
IF n < x-1 THEN Search$=Search$ & Bits$[n] & "%" ELSE Search$=Search$ & Bits$[n] ENDIF
ENDIF
NEXT
HTML$ = EXEC$(Search$) IF NOT(INSTR(HTML$,"<a")) THEN PRINT " Can not Resolve Host : DuckDuckGo" PRINT " Check Internet Connection" END END IF 'Clean Up and save to file
SPLIT HTML$ BY "\n" TO Res$ SIZE x
FOR n = 0 TO x-1
Res$[n]= CHOP$ (Res$[n]) IF LEN(Res$[n])>0 THEN tem$ = LEFT$(Res$[n],1) IF tem$ == "<" THEN Results$ = CHOP$(Results$ & Res$[n]) & "\n" END IF END IF
NEXT
SAVE Results$ TO CURDIR$ & "/index.html" IF FILEEXISTS(CURDIR$&"/index.html") THEN
PRINT "Results saved in --> " & CURDIR$&"/index.html" Result$ = ""
OPEN CURDIR$&"/index.html" FOR READING AS myfile WHILE NOT(ENDFILE(myfile)) DO READLN txt$ FROM myfile txt$ = CHOP$(txt$) IF NOT(ENDFILE(myfile)) THEN IF LEFT$(txt$,2) == "<a" THEN IF (INSTR (txt$,"Feedback")) AND (lapptr>0) THEN DECR tags
END IF 'http://baconbashtexas.com/%3Fproduct%3D45 'baconbashtexas.com/?product=45
txt$ = REPLACE$(txt$,"&;","'") txt$ = REPLACE$(txt$,"'","'") txt$ = REPLACE$(txt$,"</b>","") txt$ = REPLACE$(txt$,"<b>","") txt$ = REPLACE$(txt$,"</a>","") txt$ = REPLACE$(txt$,"%3A",":") txt$ = REPLACE$(txt$,"%2F","/") txt$ = REPLACE$(txt$,"%2D","-") txt$ = REPLACE$(txt$,"%3D","=") txt$ = REPLACE$(txt$,"%3F","?") txt$ = REPLACE$(txt$,"%23","#") IF INSTR(txt$,"nofollow") THEN HTMLTAG$ = LEFT$(txt$,INSTR(txt$,">")) reset = 1 END IF
IF (INSTR(txt$,"nofollow" )) THEN
INCR tags
END IF
IF LEN( MID$(txt$,INSTR(txt$,">")+1)) THEN IF reset THEN PRINT "title=" , MID$(txt$,INSTR(txt$,">")+1) reset=0 ELSE PRINT "snippet=" , MID$(txt$,INSTR(txt$,">")+1) END IF END IF IF INSTR(txt$,"result__url") THEN HTMLTAG$ =MID$(HTMLTAG$,INSTR(HTMLTAG$,"http")) HTMLTAG$= LEFT$(HTMLTAG$,LEN(HTMLTAG$)-2) PRINT "href=" & HTMLTAG$ ALLHTMLTAGS$=ALLHTMLTAGS$ & HTMLTAG$ & "\n" END IF END IF ENDIF WEND CLOSE FILE myfile END IF
This is the calling program IF have uo to date terminal , right click on the line beginning with "href=" to Copy URL
' program getsearch .. calls /usr/bin/baconduckduck
LOCAL Inp$ = " "
LABEL start CLEAR Inp$ = " "
INPUT " q To Quit or Seach for >", Ino$ Ino$=CHOP$(Ino$) IF Ino$=="q" THEN END END IF IF Ino$=="Q" THEN END END IF
IF NOT(LEN(Ino$)) THEN A$=EXEC$ ("baconduckduck") ELSE A$=EXEC$ ("baconduckduck " & Ino$) END IF
PRINT A$ PRINT "******************************************************" PRINT INPUT "Press Key to continue >" ,key$ GOTO start
BR + have fun Alex
|
|
|
Post by alexfish on Jul 3, 2017 3:02:05 GMT 1
Hi All
It is easy to spot the ones to avoid
BR Alex
|
|
|
Post by alexfish on Jul 3, 2017 10:28:46 GMT 1
Hi Vovchik Now looking at the curl.bac again . need to refresh my memory cells from API & demos as to what they did on the raspberry pi have try a few example & test with libcurl4-openssl-dev ,, so need to also look at the header files.. libcurlpp-dev c++ wrapper for libcurl (development files) yet that API was easy also phantomjs , not work the effort, segfult,segfault ,,blagh blag , so not sure what is up there BR Alex
|
|
|
Post by alexfish on Jul 6, 2017 17:02:47 GMT 1
Hi Vovchik after a bit of fiddling with the encoded urls , IE "https%3A%2F%2Fwww.matthartley.com%2Fbacon%2Dand%2Dlinux%2F" did a test of replace the % with '&#x' & append ';' svg example // test file is in the archive + have added HUG MARK example :: mark2.bac <!--https%3A%2F%2Fwww.matthartley.com%2Fbacon%2Dand%2Dlinux%2F--> <!--Replace '%' with '&#x' and append ';' --> <svg width="600" height="600"> <text x="0" y="15" fill="blue"> https://www.matthartley.com/bacon-and-linux/</text> </svg>
BR Alex To All need a svg compatible viewer like 'rsvg-view-3' from svg bin tools Attachments:text.svg.bz2 (231 B)
mark2.bac.bz2 (209 B)
|
|
|
Post by vovchik on Jul 6, 2017 17:10:58 GMT 1
Dear Alex,
Yep, that works and looks perfect.
With kind regards, vovchik
|
|
|
Post by alexfish on Jul 6, 2017 17:35:24 GMT 1
Dear Alex, Yep, that works and looks perfect. With kind regards, vovchik even better can get the translate text by GRAB$ IE A$ = GRAB$(mark) PRINT A$ BR Alex
|
|
|
Post by alexfish on Jul 7, 2017 1:06:01 GMT 1
Hi All did some further reading in the archive is a simple example of the url & text snippet from a search did not Know , can copy a label from a pop up menu , now need to look the Hug IMAGE:: that is if want graphics enable in a simple browser BR Alex Attachments:mark3.bac.bz2 (469 B)
|
|
|
Post by alexfish on Jul 8, 2017 10:58:24 GMT 1
Hi Vovchik & Peter
had a few problems with Peter's curl example
in 1, Segfaults in 2, not receiving all the data IE:: only part of the PAGE recieved
adapted code to see what I be looking at,
anyway finally arrived at this
hope it works
BR Alex
' Include the CURL context INCLUDE "curl.bac"
' We store our data in this variable DECLARE mail$
' Callback to let Curl gather data FUNCTION Save_Data(char * buffer, size_t size, size_t nmemb, void *userp) mail$ = CHOP$(buffer) mail$ = LEFT$(mail$,LEN(mail$)-1) RETURN size*nmemb
END FUNCTION
' Create the handle easyhandle = curl_easy_init()
' Set the options curl_easy_setopt(easyhandle, CURLOPT_URL, "https://duckduckgo.com/html/?q=BaconConverter") curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, Save_Data)
' Perform the GET success = curl_easy_perform(easyhandle)
' Cleanup curl_easy_cleanup(easyhandle) PRINT mail$
ADDED IF Wondering what the diff is in URL above IE the /html/ and what some of the problems lay
can try this as well
' Include the CURL context INCLUDE "curl.bac"
' We store our data in this variable DECLARE mail$
' Callback to let Curl gather data FUNCTION Save_Data(char * buffer, size_t size, size_t nmemb, void *userp) mail$ = CHOP$(buffer) mail$ = LEFT$(mail$,LEN(mail$)-1) RETURN size*nmemb
END FUNCTION
' Create the handle easyhandle = curl_easy_init()
' Set the options curl_easy_setopt(easyhandle, CURLOPT_URL, "https://duckduckgo.com/?q=BaconConverter") curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, Save_Data)
' Perform the GET success = curl_easy_perform(easyhandle)
' Cleanup curl_easy_cleanup(easyhandle) PRINT mail$
|
|
|
Post by vovchik on Jul 8, 2017 11:50:24 GMT 1
Dear Alex,
This one works (in terms of looking like an html page):
"https://duckduckgo.com/html/?q=BaconConverter"
The other URL does indeed get something, but not, it seems, what we want...
And the links presented are invalid, for some reason. Wonder why. Are we not supplying some essential CURLOPT_?
With kind regards, vovchik
|
|
|
Post by alexfish on Jul 8, 2017 13:29:43 GMT 1
Dear Alex, This one works (in terms of looking like an html page): "https://duckduckgo.com/html/?q=BaconConverter" The other URL does indeed get something, but not, it seems, what we want... And the links presented are invalid, for some reason. Wonder why. Are we not supplying some essential CURLOPT_? With kind regards, vovchik MM! started think , as far as can work out there are recursive call backs , since the data is streamed in chunks however the html call works in this case even if the html is placed in the '.com/html?q=BaconConverter"' did a recursive test , found the beginning of file can , as in chunk's of data were missing, IE on getting Last Chunk of data as far as the chunks are concern here is a fix BR Alex ' Include the CURL context INCLUDE "curl.bac"
' We store our data in this variable DECLARE mail$ DECLARE MAIL$=""
' Callback to let Curl gather data FUNCTION Save_Data(char* buffer, size_t size, size_t nmemb, void *userp)
mail$ = CHOP$(buffer) MAIL$ = MAIL$ & mail$ 'mail$ = LEFT$(mail$,LEN(mail$)-1) RETURN size*nmemb
END FUNCTION
' Clean end of html SUB Clean() IF RIGHT$(MAIL$,1) = "0" THEN MAIL$ = LEFT$(MAIL$,LEN(MAIL$)-1) END IF
END SUB
SUB Search() ' Create the handle easyhandle = curl_easy_init()
' Set the options curl_easy_setopt(easyhandle, CURLOPT_URL, "https://duckduckgo.com/html?q=BaconConverter") curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, Save_Data)
' Perform the GET success = curl_easy_perform(easyhandle)
' Cleanup curl_easy_cleanup(easyhandle) CALL Clean END SUB
CALL Search()
PRINT MAIL$
|
|
|
Post by alexfish on Jul 8, 2017 13:36:03 GMT 1
Dear Alex, Thanks. What about using "curl.bac"? It should be possible, too. I have been playing with it and can grab the bacon site but am getting errors (due to my ignorance) when I try https. With kind regards, vovchik now looking. BR Alex
|
|