Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2017-12

Is there some example of an FPC program for use in svn hook calls (email)?

10 messages · 6 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Bo Berglund

I am looking at switching from CVS to SVN as version control at my company. We have used CVS(nt) since 16 years back, but it is no longer working properly on new Windows versions (server side) so I am planning to go to subversion instead.
For CVS I wrote a Delphi mailer program (CVSMailer) a long time ago, which has been used to send commit, tag and import emails to the developers from the script hooks in CVS.

Now I am looking at the same problem for svn and this time I need it to be cross-platform, hence using FPC.
Before I start on it myself I wonder if someone here has already written such a program for svn and if so, if it is available as open source?
It should work from the post-commit hook in subversion.

Thanks.

Bo Berglund Developer in Sweden

Reply by email

Adriaan van Os
to Bo Berglund
Bo Berglund wrote 5 lines
Now I am looking at the same problem for svn and this time I need it to be cross-platform, hence using FPC. Before I start on it myself I wonder if someone here has already written such a program for svn and if so, if it is available as open source?

I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though.

Regards,

Adriaan van Os

Reply by email

Mark Morgan Lloyd
to Adriaan van Os
Adriaan van Os wrote 7 lines
Bo Berglund wrote: > Now I am looking at the same problem for svn and this time I need it> > to be cross-platform, hence using FPC.> Before I start on it myself I > wonder if someone here has already> written such a program for svn and > if so, if it is available as open> source? I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though.

With the caveat that while Sendmail is a well-respected MTA, many unix systems also provide a sendmail command as a generic mailer. On Windows Blat is an alternative.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Bo Berglund
to Adriaan van Os: I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using AssignStream,…
Adriaan van Os wrote 11 lines
Bo Berglund wrote: > Now I am looking at the same problem for svn and this time I need it > to be cross-platform, hence using FPC. > Before I start on it myself I wonder if someone here has already > written such a program for svn and if so, if it is available as open > source? I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though.

OK, I am looking for a program which will make a nice looking report email to send. In CVSMailer I am parsing a lot of data from the scripts in CVS and then build a html email showing the relevant info like who made the commit, when and on which project as well as the involved files with revisions and links to view them via VieVC. But since subversion is so different I wanted to check for an already existing solution before I start creating a program by myself.

BTW CVSMailer uses Indy for the network related stuff.

Bo Berglund Developer in Sweden

Reply by email

Bo Berglund
to Mark Morgan Lloyd: With the caveat that while Sendmail is a well-respected MTA, many unix systems also…
Mark Morgan Lloyd wrote 12 lines
On 14/12/17 10:00, Adriaan van Os wrote: > Bo Berglund wrote: >> Now I am looking at the same problem for svn and this time I need it> >> to be cross-platform, hence using FPC.> Before I start on it myself I >> wonder if someone here has already> written such a program for svn and >> if so, if it is available as open> source? > I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using > AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though. With the caveat that while Sendmail is a well-respected MTA, many unix systems also provide a sendmail command as a generic mailer. On Windows Blat is an alternative.

But *sending email* is not the problem, I have done this in many applications using Indy TIdSMTP. Instead it is getting data out oof svn and formatting these in a friendly looking email to be sent....

Anyway I started on my own svn data collector and got stuck in a very basic function...

I used the information in the wiki page below on how to execute a command line program and retrieving its output data for processing. Look at the section towards the bottom with subject "Reading large output":
http://wiki.freepascal.org/Executing_External_Programs

To get the subversion commit data one has to use svnlook, which is a command line program so I need to do this.

But when I use this approach I get no data at all, what could have gone wrong here?

When I reach the end of the repeat loop there is no data at all to process, i = 0!
But when I run the command on the command line manually I do get svn data back.
So what did I do wrong when I used the wiki solution?

var
  //Command line parameters are:
  REPOS_PATH,         //Full path to repository
  REV,                //Revision number
  TXN_NAME:  string;  //Transaction name
  DataBuf,
  Parms:   TStringList;
  svncmd: string;
  SCD: TSvnCommitData;

function GetSvnData(Command: string; Parameters: TStringList; var SvnData: TStringList): boolean;
{Command is the full path to the executable to run (<path>\svnlook.exe or <path>/svnlook
Parameters contain the command line parameters. A typical parameter set could be
info
-r
2121
<repository path on server>

 The result from the command will be stored into the SvnData
stringlist
 }
const
  BUF_SIZE = 2048; // Buffer size for reading the output in chunks
var
  AProcess     : TProcess;
  OutputStream : TStream;
  BytesRead    : longint;
  i: integer;
  Buffer       : array[1..BUF_SIZE] of byte;
begin
  Result := false;
  // Set up the process
  AProcess := TProcess.Create(nil);
  // Create a stream object to store the generated output in.
  OutputStream := TMemoryStream.Create;
  //Create the data buffer
  try
    AProcess.Executable := Command;
    // Process option poUsePipes has to be used so the output can be
captured.
    AProcess.Options := [poUsePipes];
    for i := 0 to Parameters.Count -1 do
      AProcess.Parameters.Add(Parameters[i]);
    i := 0;
    // Start the process (run the command)
    AProcess.Execute;
    // All generated output from AProcess is read in a loop until no
more data is available
    repeat
      // Get the new data from the process to a maximum of the buffer
size that was allocated.
      // Note that all read(...) calls will block except for the last
one, which returns 0 (zero).
      BytesRead := AProcess.Output.Read(Buffer, BUF_SIZE);
      // Add the bytes that were read to the stream for later usage
      OutputStream.Write(Buffer, BytesRead);
      Inc(i, BytesRead);
    until BytesRead = 0;  // Stop if no more data is available
    OutputStream.Position := 0; // Required to make sure all data is
copied from the start
    SvnData.LoadFromStream(OutputStream);
    Result := (i > 0); //(SvnData.Count > 0);
  finally
    OutputStream.Free;;
    AProcess.Free;
  end;
end;
begin
  //Check for the call parameters:
  if ParamCount() < 3 then
  begin
    Writeln('Call parameters missing!');
    exit;
  end;
  REPOS_PATH := ParamStr(1);
  REV := ParamStr(2);
  TXN_NAME := ParamStr(3);
  Parms := TStringList.Create;
  DataBuf := TStringList.Create;
  SCD := TSvnCommitData.Create;
  try
    // The commands for Windows and *nix are different hence the
$IFDEFs
    {$IFDEF Windows}
      svncmd := 'C:\Programs\Subversion\svnlook.exe';
    {$ENDIF Windows}
    {$IFDEF Unix}
      svncmd := '/usr/bin/svnlook';
    {$ENDIF Unix}
    //Build the call for svnlook
    {1. Get the starting information on the operation
        svnlook info -r <REV> <REPOS_PATH>
    }
    Parms.Add('info');
    Parms.Add('-r');
    Parms.Add('REV');
    Parms.Add('REPOS_PATH');
    if GetSvnData(svncmd, Parms, DataBuf) then
    begin
      SCD.UserName := DataBuf[0];
      SCD.CommitTime := DataBuf[1];
      SCD.Logmessage := DataBuf[3];
    end;
  finally
    SCD.Free;
    DataBuf.Free;
  end;
end.
Bo Berglund Developer in Sweden

Reply by email

Paul Breneman
to Bo Berglund
Bo Berglund wrote 39 lines
On Thu, 14 Dec 2017 10:26:22 +0000, Mark Morgan Lloyd wrote: > On 14/12/17 10:00, Adriaan van Os wrote: >> Bo Berglund wrote: >>> Now I am looking at the same problem for svn and this time I need it> >>> to be cross-platform, hence using FPC.> Before I start on it myself I >>> wonder if someone here has already> written such a program for svn and >>> if so, if it is available as open> source? >> I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using >> AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though. > > With the caveat that while Sendmail is a well-respected MTA, many unix > systems also provide a sendmail command as a generic mailer. On Windows > Blat is an alternative. But *sending email* is not the problem, I have done this in many applications using Indy TIdSMTP. Instead it is getting data out oof svn and formatting these in a friendly looking email to be sent.... Anyway I started on my own svn data collector and got stuck in a very basic function... I used the information in the wiki page below on how to execute a command line program and retrieving its output data for processing. Look at the section towards the bottom with subject "Reading large output": http://wiki.freepascal.org/Executing_External_Programs To get the subversion commit data one has to use svnlook, which is a command line program so I need to do this. But when I use this approach I get no data at all, what could have gone wrong here? When I reach the end of the repeat loop there is no data at all to process, i = 0! But when I run the command on the command line manually I do get svn data back. So what did I do wrong when I used the wiki solution?

I recently used the same code, so I tried to execute your code (below). I'd suggest adding the two lines (shown below) to troubleshoot. I didn't execute svnlook but ppcx64 with parameter -h and that works okay with your code.

Show quoted text 73 lines
var //Command line parameters are: REPOS_PATH, //Full path to repository REV, //Revision number TXN_NAME: string; //Transaction name DataBuf, Parms: TStringList; svncmd: string; SCD: TSvnCommitData; function GetSvnData(Command: string; Parameters: TStringList; var SvnData: TStringList): boolean; {Command is the full path to the executable to run (<path>\svnlook.exe or <path>/svnlook Parameters contain the command line parameters. A typical parameter set could be info -r 2121 <repository path on server> The result from the command will be stored into the SvnData stringlist } const BUF_SIZE = 2048; // Buffer size for reading the output in chunks var AProcess : TProcess; OutputStream : TStream; BytesRead : longint; i: integer; Buffer : array[1..BUF_SIZE] of byte; begin Result := false; // Set up the process AProcess := TProcess.Create(nil); // Create a stream object to store the generated output in. OutputStream := TMemoryStream.Create; //Create the data buffer try AProcess.Executable := Command; // Process option poUsePipes has to be used so the output can be captured. AProcess.Options := [poUsePipes]; for i := 0 to Parameters.Count -1 do AProcess.Parameters.Add(Parameters[i]); i := 0; // Start the process (run the command) AProcess.Execute; // All generated output from AProcess is read in a loop until no more data is available repeat // Get the new data from the process to a maximum of the buffer size that was allocated. // Note that all read(...) calls will block except for the last one, which returns 0 (zero). BytesRead := AProcess.Output.Read(Buffer, BUF_SIZE); // Add the bytes that were read to the stream for later usage OutputStream.Write(Buffer, BytesRead); Inc(i, BytesRead); until BytesRead = 0; // Stop if no more data is available OutputStream.Position := 0; // Required to make sure all data is copied from the start SvnData.LoadFromStream(OutputStream); Result := (i > 0); //(SvnData.Count > 0);
// I'd suggest adding these two lines to troubleshoot:
     writeln(SvnData.Text);
     writeln('Number of bytes = ', i);
Show quoted text 51 lines
finally OutputStream.Free;; AProcess.Free; end; end; begin //Check for the call parameters: if ParamCount() < 3 then begin Writeln('Call parameters missing!'); exit; end; REPOS_PATH := ParamStr(1); REV := ParamStr(2); TXN_NAME := ParamStr(3); Parms := TStringList.Create; DataBuf := TStringList.Create; SCD := TSvnCommitData.Create; try // The commands for Windows and *nix are different hence the $IFDEFs {$IFDEF Windows} svncmd := 'C:\Programs\Subversion\svnlook.exe'; {$ENDIF Windows} {$IFDEF Unix} svncmd := '/usr/bin/svnlook'; {$ENDIF Unix} //Build the call for svnlook {1. Get the starting information on the operation svnlook info -r <REV> <REPOS_PATH> } Parms.Add('info'); Parms.Add('-r'); Parms.Add('REV'); Parms.Add('REPOS_PATH'); if GetSvnData(svncmd, Parms, DataBuf) then begin SCD.UserName := DataBuf[0]; SCD.CommitTime := DataBuf[1]; SCD.Logmessage := DataBuf[3]; end; finally SCD.Free; DataBuf.Free; end; end.

Reply by email

Marco van de Voort
to Bo Berglund: On Thu, 14 Dec 2017 10:26:22 +0000, Mark Morgan Lloyd But *sending email* is not the…

In our previous episode, Bo Berglund said:

But *sending email* is not the problem, I have done this in many applications using Indy TIdSMTP. Instead it is getting data out oof svn and formatting these in a friendly looking email to be sent....

Actually process.runcommand was developed to get the output of SVN commands.

It is used to create the mergelogs http://www.stack.nl/~marcov/mergelogs26/

Reply by email

Bo Berglund
to Marco van de Voort
Show quoted text 8 lines
In our previous episode, Bo Berglund said: > But *sending email* is not the problem, I have done this in many > applications using Indy TIdSMTP. Instead it is getting data out oof > svn and formatting these in a friendly looking email to be sent.... Actually process.runcommand was developed to get the output of SVN commands. It is used to create the mergelogs http://www.stack.nl/~marcov/mergelogs26/

Thanks for your suggestion!

Do you mean replace the current code:

    AProcess.Executable := Command;
    AProcess.Options := [poUsePipes];
    for i := 0 to Parameters.Count -1 do
      AProcess.Parameters.Add(Parameters[i]);
    AProcess.Execute;
    repeat
      BytesRead := AProcess.Output.Read(Buffer, BUF_SIZE);
      OutputStream.Write(Buffer, BytesRead);
    until BytesRead = 0;  // Stop if no more data is available
    OutputStream.Position := 0;
    SvnData.LoadFromStream(OutputStream);

With something like this:

   if AProcess.RunCommand(Command, Parameters.Strings, sTmp) then
   begin
     SvnData.Text := sTmp);
     Result := true;
   end;

Are all of those preliminaries unnecessary and could be discarded then?

BTW:
How can I view what is actually happening in the command window when debugging the code?
When I started this (in Lazarus 1.6) I selected a new "Program" rather than a "Console application".

Bo Berglund Developer in Sweden

Reply by email

Giuliano Colla
to Bo Berglund: On Thu, 14 Dec 2017 10:26:22 +0000, Mark Morgan Lloyd But *sending email* is not the…

Il 14/12/2017 18:02, Bo Berglund ha scritto:

But*sending email* is not the problem, I have done this in many applications using Indy TIdSMTP. Instead it is getting data out oof svn and formatting these in a friendly looking email to be sent....

You might find useful guidelines in the fpcup and fpcupdeluxe programs which extensively use svn to automatically update the fpc and lazarus versions from svn:

https://github.com/LongDirtyAnimAlf/Reiniero-fpcup

https://github.com/newpascal/fpcupdeluxe

(btw they provide also a comfortable way to keep different fpc and lazarus versions on the same platform, to set them up for crosscompiling, etc.)

Giuliano

Reply by email

Bo Berglund
to Bo Berglund: On Thu, 14 Dec 2017 19:37:55 +0100 (CET), Thanks for your suggestion! Do you mean replace…
Bo Berglund wrote 6 lines
if AProcess.RunCommand(Command, Parameters.Strings, sTmp) then begin SvnData.Text := sTmp); Result := true; end;
It works when this is changed to:
var
  Params: array of string;
  ...
begin
....
    SetLength(Params, Parameters.Count);
    for i := 0 to Parameters.Count-1 do
      Params[i] := Parameters[i];
    if RunCommand(Command, Params, sTmp) then
    begin
      SvnData.Text := sTmp;
      Result := true;
      writeln(SvnData.Text); <= Just for testing
    end;

But first I had to fix the real bug:

*** WHAT A STUPID MISTAKE! BANGING MY HEAD! ***
In the caller:

    Parms.Add('info');
    Parms.Add('-r');
    Parms.Add('REV');
    Parms.Add('REPOS_PATH');
    if GetSvnData(svncmd, Parms, DataBuf) then

The last two Parms.Add's are stupidly wrong. The quotes are leftovers from when I had literals there as a first test. REV and REPOS_PATH are *variables*! Removing the quotes is what it took to make it work....

But a good thing came out of this anyway because I was steered towards the use of RunCommand(), which works just fine and simplifies the code enormously!
Thanks for that!

Bo Berglund Developer in Sweden

Reply by email