I have written a Lazarus/FPC program to configure an IoT device via a tcp server I have implemented on the device. It all works well for setting the parameters etc.
Now I want to also add a firmware update function to this configurator program, but I could use some advice regarding the way to send the file data...

The firmware updater service is running on the IoT device on port 80 as a web server and when one connects to that URL with a browser the following happens:

1) A login dialog is shown where a username and a password needs to be entered.

2) Next a very simple web form is shown with the following source: <html>
<body>
<form method='POST' action='' enctype='multipart/form-data'> <input type='file' name='update'>
<input type='submit' value='Update'>
</form>
</body>
</html>

3) When the "Browse..." button is clicked one gets to navigate the file system and select a file to upload. The name is then displayed in a box.

4) Upon hitting the submit button "Update" the file is sent to the IoT device and used to update the firmware.

Now I want to get rid of the web browser step and integrate the functionality of the form into the configuration program, but I don't know how to actually send the POST data to the IoT device so it will work...

The instructions for the WebUpdater service I have added to the IoT device has a second alternative for uploading the firmware using curl:

curl -u user:pwd -F "" <ip-address>:80/firmware

I don't want to require that curl is present on the PC where my program will run, though, so I want to implement the actual post message in my fpc code.

Does someone have a suggestion on how to do this?

By the way, I am using the Indy10 components via the indylaz package. Lazarus is 1.8.0 and FPC 3.0.4 on Windows7.

El 04/04/18 a les 13:50, Bo Berglund ha escrit:

I never used indy, but you can either use TFPHTTPClient (unit fphttpclient) or synapse THHPSend.
Both allow to send form data.
Disregard my suggestion if you prefer to use indy.

Bye

Yes, I use Indy since the Delphi days but I have not used it to send files before.
Indy has a TIdHTTP object/component, which presumably can be used but there are no good examples I have found that show how to use it for simulating POST from a form like the one I showed...

I don't know what a 'good example' is for you, but the code in

https://stackoverflow.com/questions/27257577/indy-mime-decoding-of-multipart-form-data-requests-returns-trailing-cr-lf

Is quite understandable.

Michael.

I found this example too, but it seems to be dealing with *server* side stuff...

I am trying to create a *client* function which will send a POST command like a browser would when I click the submit button on the simple form I showed the source for.

It works fine if I use the web browser to connect and load the form, then select the file and finally hit the submit button. But since I already have all of the other device config items inside an FPC program I wanted the firmware upload to also be part of it...

This is how far I have gotten with the UploadFirmware method:

function TConfigCommHandler.UploadFirmware(FileName: string; URL:
string): boolean;
var
  HTTP: TIdHTTP;
  Src: TIdMultipartFormDataStream;
begin
  Result := false;
  if not FileExists(FileName) then exit;
  HTTP := TIdHTTP.Create;
  Src := TIdMultipartFormDataStream.Create;
  try
    Src.AddFormField('?',?);
    Src.AddFile(FileName,'?');
    try
      HTTP.Post(URL, Src, ?);
      Result := true;
    except
      on E: Exception do
        FLastError := 'Exception: ' + E.Message;
    end;
  finally
    Src.Free;;
    HTTP.Free;
  end;
end;

It is all about formulating the correct parameters towards the HTTP.Post function.

HTTP is quite simple actually. You just build a post message (a string), and send it over a socket. I've implemented HTTP client functions in many computer languages. Here are the key take aways if you want to implement it yourself.

Build this string filling out the blanks using your own values

POST {location} HTTP 1.1{linefeed}
Host: {domain}{linefeed}
Content-Length: {length}{linefeed}
{linefeed}
{content}

So to post the word 'hello' to http://example.com/yourpage?say, where the domain is example.com, and location is yourpage?say, the string would look like this in pascal.

Message :=
'POST /yourpage?say HTTP 1.1'#10 +
'Host: example.com'#10 +
'Content-Length: 5'#10 +
#10 +

  'hello';

Then you use some socket component, set the Host to 'example.com' so that it can resolve the ip address, and call Send(Message).

This assumes you have a tcp client socket component with properties Host, and a method called Send, but most any/every socket component does.

If you want to post form named arguments in content, then just format them like:

Content := 'words=hello&priority=high&account=123';
ContentLength := IntToStr(Length(Content));

Message :=
'POST /yourpage?say HTTP 1.1'#10 +
'Host: example.com'#10 +
'Content-Length: ' + ContentLength + #10 +
#10 +

  Content;

Socket.Host := 'example.com';
Socket.Send(Message);

It's that easy.

Just a follow up, if care you get a response back from your POST, as in the server says okay, you just use the same socket to read. Also, you might need to call connect after you set the host:

var
  Message: string;
  Response: string;
...
// port 80 normally used for http
Socket.Connect('example.com', 80);
Socket.Send(Message);
Socket.Read(Response);
WriteLn(Response);

If the server accepted your POST, the response will start with 'HTTP 1.1 200 OK'#10, followed by some other headers. Socket Read might be named Receive or Recv, depending on your client socket implementation of choice.

If you want to learn may, you may want to example my THttpClient object:

https://github.com/sysrpl/Cross.Codebot/blob/master/source/codebot.networking.web.pas#L532

And also, here is an example of how to write an object oriented socket class that wraps the traditional socket apis on all platforms:

https://github.com/sysrpl/Cross.Codebot/blob/master/source/codebot.networking.pas#L84

If you look at that source you can also see how simple it is to add https support to your socket.

Socket.Secure := True;
Socket.Connect('example.com', 443);

And finally, you don't need to use my github code. anyAclient socket class for Pascal should work approximately the same.

Bo Berglund wrote

Yes, I use Indy since the Delphi days but I have not used it to send files before.

Yes, there is no problem in using Indy with FPC - I've done it for years. So no need to switch components.

Indy has a TIdHTTP object/component, which presumably can be used but there are no good examples

Have you actually tried asking the question in the official Indy support newsgroup?

NNTP Server: news.atozed.com
Groups: atozedsoftware.indy.general
atozedsoftware.indy.fpc
atozedsoftware.indy.protocol.*
atozedsoftware.indy.non-tech
... and many others....

I've always gotten very quick responses from them.

Have you tried this?

http://lmgtfy.com/?q=delphi+indy+sending+file+via+http+post

I see many answers listed there. ;-)

Regards,
Graeme

And if you are really lazy... ;-)

Indy has TIdMultipartFormDataStream for this purpose:

procedure TForm1.SendPostData;
var
  Stream: TStringStream;
  Params: TIdMultipartFormDataStream;
begin
  Stream := TStringStream.Create('');
  try
   Params := TIdMultipartFormDataStream.Create;
   try
    Params.AddFile('File1', 'C:\test.txt','application/octet-stream');
    try
     HTTP.Post('http://posttestserver.com/post.php', Params, Stream);
    except
     on E: Exception do
       ShowMessage('Error encountered during POST: ' + E.Message);
    end;
    ShowMessage(Stream.DataString);
   finally
    Params.Free;
   end;
  finally
   Stream.Free;
  end;
end;

That will sent a file to the PHP script on the server.

Regards,
Graeme

Graeme Geldenhuys wrote

And if you are really lazy... ;-)

And here's an even simpler version.

-------- [ from Remy Lebeau ]-------------------------

TIdHTTP has two overloaded versions of Post() that take a filename as input:

var
  Response: String;
Response :=
IdHTTP1.Post('https://www.example.com/ex/exampleAPI.asmx/Process',
'c:\filename.txt');

....or....

var
  Response: TStream;
Response := TMemoryStream.Create;
IdHTTP1.Post('https://www.example.com/ex/exampleAPI.asmx/Process',
'c:\filename.txt', Response);
...
Response.Free;

Note that you are posting to an HTTPS URL, so you need to first assign an SSL-enabled IOHandler, such as TIdSSLIOHandlerSocketOpenSSL, to the TIdHTTP.IOHandler property beforehand.

-------------------[ end ]------------------------

Can't get simpler than that!

Regards,
Graeme

Well, I asked in the Embarcadero forum (using XanaNews nttp interface...) in the internet.winsock subforum and I got a reply from Remy Lebeau (as always helpful).

So I think I am good on the POST item, but I have not been able to test yet due to travels. Will try my hand on that shortly.

I can post the final solution here after I have completed the Lazarus config program with the new function and verified that it works.

Those newsgroups are a horrid suggestion. The most recent messages are from 2016. It seems they only leave it up for posterity.

Alexander Grotewohl wrote

Those newsgroups are a horrid suggestion. The most recent messages are from 2016. It seems they only leave it up for posterity.

I got personally assured by Remy Lebeau that the NNTP newsgroups are checked just as frequently as the Web Forums. The two are not synced, but both are active options for Indy support.

ps:
My last question in the Indy newsgroup (*.protocol.imap4 group) was in 2017 and if you looked closer, you will see those got answered the same day.

ps #2:
I much prefer NNTP newsgroups to the much slower and often ad-driven web forums, or mailing lists. Plus I can easily download all messages (dating back to 2006) for offline viewing and searching with my favourite NNTP client of choice.

Regards,
Graeme

Graeme Geldenhuys wrote

I much prefer NNTP newsgroups to the much slower and often ad-driven web forums, or mailing lists. Plus I can easily download all messages (dating back to 2006) for offline viewing and searching with my favourite NNTP client of choice.

Same here!
(btw I dropped off this topic because I had a fall from the roof of my house and am currently hospitalized. Got my laptop yesterday though...)

Bo Berglund wrote

By the way, I am using the Indy10 components via the indylaz package. Lazarus is 1.8.0 and FPC 3.0.4 on Windows7.

If anybody wants to do this just with winsock:

const
   dstport = 80;
   http_path = '/post.php';
type
   Int32 = Longint;
function wordstr (value:word) : string;
var
   c : string;
begin
   str (value,c);
   wordstr := c;
end;
procedure SendHttpPost (c:ansistring;
                         var msg:string);
var
   wsadata : TWSAData;  // winsock
var
   sock : TSocket;
   addr : sockets.sockaddr_in;
   numbytes : Int32;
   sendstring : ansistring;
   rc : longint;
   ok : boolean;
begin
   // initialize winsock
   if (WSAStartup($0202, wsadata)  0) then begin
      msg := 'cannot initialize winsock';
      exit;
   end;
   // get socket
   sock := fpSocket(AF_INET, SOCK_STREAM, 0);
   if (sock = -1) then begin
      msg := 'cannot retrieve socket';
      exit;
   end;
   addr.sin_family := AF_INET;
   addr.sin_port := htons(dstport);
   GetServerIP (addr.sin_addr, ok);
   rc := fpConnect (sock, @addr, sizeof(addr));
   if rc  0 then begin
      msg := 'cannot connect to host';
      exit;
   end;
   // c := stringreplace (c,' ','&',[rfReplaceAll]);
   // send data
   sendstring := 'POST '+http_path+' HTTP/1.1'+#13+#10
               + 'host: '+NetAddrToStr(addr.sin_addr)+#13+#10
               + 'Content-Type: application/x-www-form-urlencoded'+#13+#10
               + 'Content-Length: '+wordstr(length(c))+#13+#10
               + #13+#10
               + c;
   numbytes := fpSend(sock, @sendstring[1], length(sendstring), 0);
   if (numbytes = SOCKET_ERROR) then begin
      msg := 'cannot send data to host';
   end else begin
      msg := 'Sent '+wordstr(numbytes)+' bytes to
'+NetAddrToStr(addr.sin_addr));
   end;
   // clean up
   closesocket(sock);
   // shut down winsock
   WSACleanup;
end;

I'd strongly recommend using SSH/SCP instead, or at least HTTPS. Certificates for HTTPS can be more difficult to set up on nontechnical user's computers.

Cheers,
R0b0t1