Parsing web log files
Modified 2003-02-18
Thanks to Derek J. Kalweit for pointing out this technique. In VFP 6.0 (at
least) and above, if you have a class with the same name in two class libraries,
you can refer to the class with the classlib prefixing the class name in the
CREATEOBJECT() function.
loTxt1 = CREATEOBJECT("Textbox")
loTxt1.SaveAsClass("alias.vcx", "txtAlias")
loTxt2 = CREATEOBJECT("Textbox")
loTxt2.Width = loTxt2.Width * 2
loTxt2.SaveAsClass("alias2.vcx", "txtAlias")
SET CLASSLIB TO alias, alias2
loTxtAlias = CREATEOBJECT("alias.txtAlias")
?loTxtAlias.Width && 100
loTxtAlias2 = CREATEOBJECT("alias2.txtAlias")
?loTxtAlias2.Width && 200
SET CLASSLIB TO
*!* The NEWOBJECT function allows you to remove any ambiguity
*!* between the classlibs, as well as permitting the use of
*!* PRG class definitions as well.
loTxtAlias = NEWOBJECT("txtAlias", "alias.vcx")
? loTxtAlias.Width
loTxtAlias2 = NEWOBJECT("txtAlias", "alias2.vcx")
? loTxtAlias2.Width
*!* The following syntax for creating the class definition will
*!* only work in VFP7. The NEWOBJECT call, however, will work in 6,
*!* if you create the .prg manually.
TEXT TO lcClassDef NOSHOW
DEFINE CLASS txtAlias AS TextBox
Width = 300
ENDDEFINE
ENDTEXT
STRTOFILE(lcClassDef, "alias3.prg")
loTxtAlias3 = NEWOBJECT("txtAlias", "alias3.prg")
? loTxtAlias3.Width && 300
Top
The following code will help you determine if you're connected to the internet. However,
for connections that are "always on", it may tell you you're online, even if you actually
aren't.
DECLARE SHORT InternetGetConnectedState IN wininet ;
LONG @ lpdwFlags, LONG reserved
lnFlags = 0
reserved = 0
? InternetGetConnectedState(@lnFlags, reserved)
? lnFlags
lnFlags consists of the following constants ORed
together.
#define INTERNET_CONNECTION_MODEM 0x01
#define INTERNET_CONNECTION_LAN 0x02
#define INTERNET_CONNECTION_PROXY 0x04
#define INTERNET_CONNECTION_MODEM_BUSY 0x08 /* no longer used */
#define INTERNET_RAS_INSTALLED 0x10
#define INTERNET_CONNECTION_OFFLINE 0x20
#define INTERNET_CONNECTION_CONFIGURED 0x40
You can find them in WinInet.h, which installs with Visual C++.
If all else fails, the following code will work, if inelegantly (as long as your
firewall allows ping packets through).
RUN ping www.google.com > c:\ping.txt
lcPingResult = FILETOSTR("c:\ping.txt")
?AT("Reply", lcPingResult) != 0
Top
Created 2003-02-18
Randy Pearson put together an
excellent matrix explaining how different types of error handling work together
(TRY...CATCH, Object.Error, ON ERROR). However, there was one thing he left out: error handling
across process boundaries, from a COM DLL to a VFP app. The code below shows several variations
of error handling, ending with an example of how you might use COMRETURNERROR with structured
error handling to return a THROWn error from an MTDLL.
#DEFINE vfpCRLF CHR(13) + CHR(10)
LOCAL lcOldSafety, lcThrowTest, lcThrowTest2, loTT1, loTT1a, loTT2, loTT2a, ;
loErr as Exception, loCaught as Exception, llNoError
lcOldSafety = SET("Safety")
SET SAFETY OFF
*!* Here we create the class that intentionally THROWs an error
*!* in its BadCall method.
***************************START TEXT***************************
TEXT TO lcThrowTest NOSHOW
#DEFINE vfpCRLF CHR(13) + CHR(10)
DEFINE CLASS ThrowTest AS SESSION OLEPUBLIC
*!* PROCEDURE Error
*!* LPARAMETERS nError, cMethod, nLine
*!* IF INLIST(_VFP.StartMode, 3, 5)
*!* COMRETURNERROR("DLL Error Event", ;
*!* "Error " + TRANSFORM(nError) + " in " + cMethod + ", line " + TRANSFORM(nLine))
*!* ELSE
*!* MESSAGEBOX("ERROR METHOD" + vfpCRLF + vfpCRLF + ;
*!* "Error number: " + TRANSFORM(nError) + vfpCRLF + ;
*!* "Method: " + TRANSFORM(cMethod) + vfpCRLF + ;
*!* "Line Number: " + TRANSFORM(nLine) + vfpCRLF)
*!* ENDIF
*!* ENDPROC
PROCEDURE BadCall
LOCAL loErr AS Exception
TRY
ERROR 1
CATCH TO loErr
IF INLIST(_VFP.StartMode, 3, 5)
THIS.Error(loErr.ErrorNo, loErr.Procedure, loErr.LineNo)
ELSE
THROW loErr
ENDIF
ENDTRY
ENDPROC
ENDDEFINE
ENDTEXT
****************************END TEXT****************************
STRTOFILE(lcThrowTest, "throwTest.prg")
loTT1 = NEWOBJECT("ThrowTest", "throwTest.prg")
*!* First, we'll just call it as a local object, with only TRY...CATCH.
TRY
loTT1.BadCall()
CATCH TO loErr
loCaught = loErr.UserValue
WITH loCaught
MESSAGEBOX("CAUGHT ERROR" + vfpCRLF + vfpCRLF + ;
"Error number: " + TRANSFORM(.ErrorNo) + vfpCRLF + ;
"Message: " + TRANSFORM(.Message) + vfpCRLF + ;
"Line Number: " + TRANSFORM(.LineNo) + vfpCRLF + ;
"Line Contents: " + .LineContents)
ENDWITH
ENDTRY
*!* Now, we'll compile it into an MTDLL, and see what we can catch here.
BUILD PROJECT ThrowTest FROM throwTest.prg
BUILD MTDLL ThrowTest FROM ThrowTest
loTT1a = CREATEOBJECT("ThrowTest.ThrowTest")
llNoError = .T.
TRY
loTT1a.BadCall()
CATCH TO loErr
MESSAGEBOX("This code won't execute: TRY...CATCH won't go across " + ;
"the process boundary.")
llNoError = .F.
ENDTRY
IF llNoError
MESSAGEBOX("No error was propagated when THROWing from an MTDLL.")
ENDif
*!* Now, let's uncomment the Error method, and see how that affects matters
*!* in each case.
lcThrowTest2 = STRTRAN(lcThrowTest, "*!*", "")
STRTOFILE(lcThrowTest2, "throwTest2.prg")
loTT2 = NEWOBJECT("ThrowTest", "throwTest2.prg")
TRY
loTT2.BadCall()
CATCH TO loErr
MESSAGEBOX("This code won't run: the Error method will trap the error.")
ENDTRY
BUILD PROJECT ThrowTest2 FROM throwTest2
BUILD MTDLL ThrowTest2 FROM ThrowTest2
loTT2a = CREATEOBJECT("ThrowTest2.ThrowTest")
TRY
loTT2a.BadCall()
CATCH TO loErr
WITH loErr
MESSAGEBOX("CAUGHT ERROR" + vfpCRLF + vfpCRLF + ;
"Error number: " + TRANSFORM(.ErrorNo) + vfpCRLF + ;
"Message: " + TRANSFORM(.Message) + vfpCRLF + ;
"Line Number: " + TRANSFORM(.LineNo) + vfpCRLF + ;
"Line Contents: " + .LineContents)
ENDWITH
ENDTRY
SET SAFETY &lcOldSafety
Top
Created 2003-06-03
There is an optimization in VFP string handling that allows blazingly fast
work with long strings. However, this can cause problems with recursive
algorithms that build long strings. To get around it, instead of doing
lcString = lcString + lcMyNewText
you can do
lcString = "" + lcString + lcMyNewText
That extra null string disables the optimization, so that the normal string
handling kicks in, and the recursion doesn't get confused.
For an example of this in action, check out
http://www.capescience.com/webservices/globalweather/GWFoxpro.html - that's
the code where I found this bugbehavior.
You'll see near the end, where I needed to use lcRetVal = "" + lcRetVal...
If you want to try commenting out the null string, feel free. The problem may or
may not trigger, depending on the data that's coming back from the web service.
The actual behavior, as I remember it, is that once you get past a certain
level of the recursion, the information on the stack is overwritten by the last
data to be added to the string. It's hard to explain properly without showing
lengthy examples, so play with the code yourself, until you feel comfortable
with your understanding.
Top
Created 2004-03-03
I came up with this routine after typing out a SELECT statement too often. For details on how to add
a new custom Intellisense routine to VFP, see the help files.
LPARAMETERS oFoxCode
#DEFINE vfpCRLF CHR(13) + CHR(10)
#DEFINE vfpTAB CHR(9)
#DEFINE vfpINDENT SPACE(7)
LOCAL i, lnFieldCount, lcRetVal, lcField, lcTable, lcAlias
lcRetVal = ""
oFoxCode.ValueType = "V"
lcTable = GETFILE("DBF")
*!* If you don't select a table in GETFILE(), use the current table.
IF EMPTY(lcTable)
lcTable = ALIAS()
ENDIF
IF EMPTY(lcTable)
RETURN ""
ELSE
IF NOT USED(JUSTSTEM(lcTable))
USE (lcTable) IN 0
ENDif
ENDIF
lcAlias = ALIAS(JUSTSTEM(lcTable))
lnFieldCount = FCOUNT(lcAlias)
lcRetVal = "SELECT "
*!* I needed to pad all the fields out to 80: you can modify it as necessary.
FOR i = 1 TO lnFieldCount - 1
lcField = LOWER(FIELD(i, lcAlias))
lcRetVal = lcRetVal + ;
"PADR(" + lcField + ", 80) AS " + lcField + ", ;" + vfpCRLF + vfpINDENT
ENDfor
lcField = LOWER(FIELD(lnFieldCount, lcAlias))
lcRetVal = lcRetVal + lcField + " AS " + lcField + " ;" + vfpCRLF
lcRetVal = lcRetVal + vfpTAB + "FROM " + LOWER(lcAlias) + " ;" + vfpCRLF
USE IN (lcAlias)
RETURN lcRetVal
Created 2004-05-10
The other day, I was trying to import the log files from
NorthshoreCoop.org, so I ran VFP's Import
Wizard. Unfortunately, it doesn't work well with the different delimiters in a typical
Apache log. So, I gave Excel's Import Wizard a
chance at it, with identical results.
The problem here is that there are three delimiters used: space, space+bracket (for date), and
space+quote. So, we need to write some code that can deal with all three of these. ALINES()
does a good job of breaking strings down: I use it first to break the log file down into
individual lines, and then use it on each line to break it down into space-delimited tokens.
The FIELD() function is also useful, as it allows me to make the code very generic, by just
letting it know which field I'm in by ordinal (first, second, third) instead of by name.
I would not be surprised if the file you're trying to parse uses a slightly different format
than I had to deal with: I couldn't find any documentation that told me what the last column
in the file was. You should be able to just tweak the CREATE CURSOR statement: there are no
hard-coded field names or counts after it.
CREATE CURSOR crsrLog ( ;
cClientIP C(15), ;
cIdentity C(20), ;
cRemoteUser C(40), ;
cDateTime C(30), ;
cRequestLine C(80), ;
cStatusCode C(3), ;
cSize C(10), ;
cDomain C(80), ;
cReferer C(80), ;
cUserAgent C(80), ;
cUnknown C(20))
lcLog = FILETOSTR(GETFILE())
*!* Read each line into an array element
lnLineCount = ALINES(laLogLines, lcLog)
FOR i = 1 TO lnLineCount
APPEND BLANK
lcLogLine = laLogLines(i)
*!* Break the line down into space-delimited tokens.
lnTokenCount = ALINES(laTokens, lcLogLine, .T., " ")
lnField = 1
lcField = ""
llInString = .F.
FOR j = 1 TO lnTokenCount
lcToken = laTokens(j)
*!* Does the token start or end with a delimiter?
llLeft = INLIST(LEFT(lcToken, 1), '"', '[')
llRight = INLIST(RIGHT(lcToken, 1), '"', ']')
DO CASE
CASE llLeft = llRight
IF llInString
*!* If we're in the middle of a string, append the token to the string.
lcField = lcField + " " + lcToken
ELSE
*!* Otherwise, just fill in the field and go to the next one.
REPLACE (FIELD(lnField)) WITH lcToken
lnField = lnField + 1
ENDIF
*!* If the field starts with a delimiter, start building the string.
CASE llLeft AND NOT llRight
llInString = .T.
lcField = lcToken
*!* If the field ends with a delimiter, finish building the string,
*!* and fill in the next field
CASE NOT llLeft AND llRight
lcField = lcField + " " + lcToken
REPLACE (FIELD(lnField)) WITH lcField
lnField = lnField + 1
llInString = .F.
ENDCASE
ENDFOR
lnField = 1
ENDfor
Top
![[Company Logo Image]](images/logo.gif){kind=logo}
