Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2020-06

Google APIs - Authenticate using a service account?

24 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Wayne Sherman

Is there support in the fpc google api units for authenticating as a service account?

Background:
Service accounts allow an application to interact with Google APIs without authenticating as a specific user. Instead a service account is created with it's own credentials, ID, and key. By authenticating as a service account, applications can perform tasks on behalf of the users in a G Suite domain without having to have their login passwords.

Here is what it looks like in python:

Create a Credentials object from the service account's credentials and the scopes your application needs access to. For example:

[python code]
from google.oauth2 import service_account

SCOPES = ['https://www.googleapis.com/auth/gmail.labels'] SERVICE_ACCOUNT_FILE = '/path/to/service.json'

credentials = service_account.Credentials.from_service_account_file( SERVICE_ACCOUNT_FILE, scopes=SCOPES)
[/python code]

Delegate domain-wide authority

If you have delegated domain-wide access to the service account and you want to impersonate a user account, use the with_subject method of an existing ServiceAccountCredentials object.
For example:

[python code]
delegated_credentials = credentials.with_subject('') [/python code]

Use the Credentials object to call Google APIs in your application.

Reference Link:
https://developers.google.com/identity/protocols/oauth2/service-account

Background Links:
https://cloud.google.com/docs/authentication/
https://cloud.google.com/iam/docs/service-accounts

(btw, *thank you* Michael Van Canneyt et al. for your work in providing the Google API support in fpc)

Reply by email

Wayne Sherman
to Wayne Sherman
PM Wayne Sherman wrote
Is there support in the fpc google api units for authenticating as a service account?

Also for reference, here is a sample G Suite Service OAuth2 json file (with made up keys and IDs). After configuring a service account, this file can be downloaded locally and used by applications to authenticate as a service:

{
"type": "service_account",
"project_id": "g-suite-email-labels-work",
"private_key_id": "9387991bed423dccc56b5dedc6676c5e81ed9b74", "private_key": "-----BEGIN PRIVATE KEY-----\<redacted base64 strings>\n-----END PRIVATE KEY-----\n",
"client_email":
"",
"client_id": "869930957598767961280",
"auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url":
"https://www.googleapis.com/robot/v1/metadata/x509/our-domain-service-acct%40g-suite-email-labels-work.iam.gserviceaccount.com" }

Reply by email

Michael Van Canneyt
to Wayne Sherman: Is there support in the fpc google api units for authenticating as a service account?…
Wayne Sherman wrote 10 lines
Is there support in the fpc google api units for authenticating as a service account? Background: Service accounts allow an application to interact with Google APIs without authenticating as a specific user. Instead a service account is created with it's own credentials, ID, and key. By authenticating as a service account, applications can perform tasks on behalf of the users in a G Suite domain without having to have their login passwords.

The problem with the service account is that you must create a JWT Token. FPC does not yet have a unit that can generate *and sign* a JWT Token.

The good news is I have this weekend a reminder that I received some code that will allow me to complete the JWT Token support in FPC using at least the RSA256 signing algorithm (and some others as well).

Now all that is needed is to clone myself so I can actually do the work...

Michael.

Reply by email

Wayne Sherman
to Michael Van Canneyt
AM Michael Van Canneyt wrote 6 lines
The problem with the service account is that you must create a JWT Token. FPC does not yet have a unit that can generate *and sign* a JWT Token. The good news is I have this weekend a reminder that I received some code that will allow me to complete the JWT Token support in FPC using at least the RSA256 signing algorithm (and some others as well).

Synopse mORMot has a JWT implementation that supports FPC: http://blog.synopse.info/post/2016/12/19/JSON-Web-Tokens-(JWT)

The blog post refers to files SynCrypto.pas and SynEcc.pas https://github.com/synopse/mORMot/

Now all that is needed is to clone myself so I can actually do the work...

Thanks for any help you can provide. :-)

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 14 lines
On Tue, Jun 30, 2020 at 12:47 AM Michael Van Canneyt wrote: > The problem with the service account is that you must create a JWT Token. > FPC does not yet have a unit that can generate *and sign* a JWT Token. > > The good news is I have this weekend a reminder that I received some code > that will allow me to complete the JWT Token support in FPC using at > least the RSA256 signing algorithm (and some others as well). Synopse mORMot has a JWT implementation that supports FPC: http://blog.synopse.info/post/2016/12/19/JSON-Web-Tokens-(JWT) The blog post refers to files SynCrypto.pas and SynEcc.pas https://github.com/synopse/mORMot/

I know. But we can't include Synopse in FPC.

Michael.

Reply by email

Wayne Sherman
to Michael Van Canneyt
AM Michael Van Canneyt wrote
I know. But we can't include Synopse in FPC.

Can just the JWT/crypto portions be incorporated if it makes sense? It is released under these Open Source licenses:

   Mozilla Public License, version 1.1 or later (MPL);
   GNU General Public License, version 2.0 or later (GPL);
   GNU Lesser General Public License, version 2.1 or later (LGPL),
with linking exception of the FPC modified LGPL.

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 6 lines
On Tue, Jun 30, 2020 at 1:36 AM Michael Van Canneyt wrote: > > I know. But we can't include Synopse in FPC. Can just the JWT/crypto portions be incorporated if it makes sense?

I'd rather not:

morMot is tightly integrated, and I would need to pull in a lot of dependencies, which I don't want.

It's not "loosely coupled, tightly integrated."
It's "tightly coupled, tightly integrated."

The code which I received now is more standalone.
It will be easier to integrate.

Michael.

Reply by email

Joost van der Sluis
to Michael Van Canneyt: The problem with the service account is that you must create a JWT Token. FPC does not…

Op 30-06-2020 om 09:47 schreef Michael Van Canneyt:

Show quoted text 16 lines
On Fri, 26 Jun 2020, Wayne Sherman via fpc-devel wrote: > Is there support in the fpc google api units for authenticating as a > service account? > > Background: > Service accounts allow an application to interact with Google APIs > without authenticating as a specific user.  Instead a service account > is created with it's own credentials, ID, and key.  By authenticating > as a service account, applications can perform tasks on behalf of the > users in a G Suite domain without having to have their login > passwords. The problem with the service account is that you must create a JWT Token. FPC does not yet have a unit that can generate *and sign* a JWT Token.

There is:

fppkg install cnocOIDC

And you will have JWT support. Have a look at the TcnocOIDCIDTokenClaim and TcnocRSAPublicKey/TcnocRSAPrivateKey classes.

Regards,

Joost.

Reply by email

Michael Van Canneyt
to Joost van der Sluis
Joost van der Sluis wrote 24 lines
Op 30-06-2020 om 09:47 schreef Michael Van Canneyt: > > On Fri, 26 Jun 2020, Wayne Sherman via fpc-devel wrote: > >> Is there support in the fpc google api units for authenticating as a >> service account? >> >> Background: >> Service accounts allow an application to interact with Google APIs >> without authenticating as a specific user.  Instead a service account >> is created with it's own credentials, ID, and key.  By authenticating >> as a service account, applications can perform tasks on behalf of the >> users in a G Suite domain without having to have their login >> passwords. > > The problem with the service account is that you must create a JWT Token. > FPC does not yet have a unit that can generate *and sign* a JWT Token. There is: fppkg install cnocOIDC And you will have JWT support. Have a look at the TcnocOIDCIDTokenClaim and TcnocRSAPublicKey/TcnocRSAPrivateKey classes.

Joost,

I am aware of that, but this kind of basic functionality *really* should be in the base distribution.

Michael.

Reply by email

Joost van der Sluis
to Michael Van Canneyt

Op 10-07-2020 om 11:02 schreef Michael Van Canneyt:

Show quoted text 11 lines
> There is: > >  fppkg install cnocOIDC > > And you will have JWT support. Have a look at the > TcnocOIDCIDTokenClaim and TcnocRSAPublicKey/TcnocRSAPrivateKey classes. Joost, I am aware of that, but this kind of basic functionality *really* should be in the base distribution.

No, googleapi should be removed from the base distribution. That makes it also easier to respond on changes and to release them more often. However this is in principle also possible with fppkg when they are in the base distribution.

Also I don't think that many other languages have JWT support in their base distribution.

To copy this into the base-distribution you also have to copy dcpcrypt. (MIT) Another solution would be to add a googleapi-plus package (outside the base distribution) that depends on both cnocRSA and the googleapi package.

I know we disagree on this. But it is really impossible to keep up with all the work that needs to be done if we have to convert all useful libraries out there into one base distribution.

Regards,

Joost.

Reply by email

Michael Van Canneyt
to Joost van der Sluis
Joost van der Sluis wrote 18 lines
Op 10-07-2020 om 11:02 schreef Michael Van Canneyt: >> There is: >> >>  fppkg install cnocOIDC >> >> And you will have JWT support. Have a look at the >> TcnocOIDCIDTokenClaim and TcnocRSAPublicKey/TcnocRSAPrivateKey classes. > > Joost, > > I am aware of that, but this kind of basic functionality *really* should > be in the base distribution. No, googleapi should be removed from the base distribution. That makes it also easier to respond on changes and to release them more often. However this is in principle also possible with fppkg when they are in the base distribution.

The JWT issue is independent of Google APIs.

Also I don't think that many other languages have JWT support in their base distribution.

FPC is more than just the language, it's a full library and run-time system.

And run-time is shifting to web:
Most desktop applications I know are being moved (or have moved) to the web.

I believe that soon, the only native apps left will be services and apps that need hardware access (automation etc.)

So I think the FPC base system should contain everything to be able to program web (without forgetting the services and hardware access, obviously), and I consider JWT to be part of that base layer.

I won't argue to move Google APIS to fppkg once the latter is fully functional. I have started to port the fppkg Web frontend to pas2js, by the way, so I hope to have it working before the summer is out and we can then put it online.

Michael.

Reply by email

Florian Klämpfl
to Michael Van Canneyt
Show quoted text 6 lines
And run-time is shifting to web: Most desktop applications I know are being moved (or have moved) to the web. I believe that soon, the only native apps left will be services and apps that need hardware access (automation etc.)

Well, using Zoom a lot the last months, it taught me that this desktop app solution is much better than any browser based video conferencing system and probably the reason why Zoom apparently won the video conferencing system race.

Reply by email

Florian Klämpfl



Show quoted text 6 lines
And run-time is shifting to web: Most desktop applications I know are being moved (or have moved) to the web. I believe that soon, the only native apps left will be services and apps that need hardware access (automation etc.)

Well, using Zoom a lot the last months, it taught me that this desktop app solution is much better than any browser based video conferencing system and probably the reason why Zoom apparently won the video conferencing system race.

Reply by email

Michael Van Canneyt
to Florian Klämpfl
Florian Klämpfl wrote 12 lines
 > > And run-time is shifting to web: > Most desktop applications I know are being moved (or have moved) to the web. > > I believe that soon, the only native apps left will be services and apps that > need hardware access (automation etc.) Well, using Zoom a lot the last months, it taught me that this desktop app solution is much better than any browser based video conferencing system and probably the reason why Zoom apparently won the video conferencing system race.

I don't know about that.
In my experience, jitsi does a good job, at least as good as Zoom.

Many web-based platforms try to correct too many things at the server level, and performance suffers from this of course. Jitsi sends streams as-is.

My point was that there is a definite shift towards web, and I think we must provide the tools to help people with that.

Which is not to say that we should abandon desktop, of course.

Michael.

Reply by email

Florian Klämpfl
to Michael Van Canneyt
Michael Van Canneyt wrote 22 lines
On Fri, 10 Jul 2020, Florian Klämpfl wrote: >  >> >> And run-time is shifting to web: >> Most desktop applications I know are being moved (or have moved) to >> the web. >> >> I believe that soon, the only native apps left will be services and >> apps that >> need hardware access (automation etc.) > > Well, using Zoom a lot the last months, it taught me that this desktop > app > solution is much better than any browser based video conferencing system > and probably the reason why Zoom apparently won the video conferencing > system race. I don't know about that. In my experience, jitsi does a good job, at least as good as Zoom.

It starts with simply things: how to use multiple screens with the web based solutions: how to undock chat window, screen cast, participants or videos and arrange them to different screens?

My point was that there is a definite shift towards web,

For me web based system are still the last resort.

Reply by email

gabor
to Florian Klämpfl

W dniu 2020-07-10 o 22:31, Florian Klämpfl pisze:

It starts with simply things: how to use multiple screens with the web based solutions: how to undock chat window, screen cast, participants or videos and arrange them to different screens?

Yes, webapps. Readability and usability less important than dark theme.

Michał.

Reply by email

Michael Van Canneyt
to gabor
gabor wrote 6 lines
W dniu 2020-07-10 o 22:31, Florian Klämpfl pisze: > It starts with simply things: how to use multiple screens with the web > based solutions: how to undock chat window, screen cast, participants or > videos and arrange them to different screens? Yes, webapps. Readability and usability less important than dark theme.

That is unfortunately not limited to webapps =-(

Do you know how many requests we get for a dark theme for Lazarus ?

Some consider it *THE* most needed improvement for Lazarus for general adoption. :-)

Michael.

Reply by email

Luca Olivetti
to Florian Klämpfl: Well, using Zoom a lot the last months, it taught me that this desktop app solution is…

El 10/7/20 a les 21:23, Florian Klämpfl ha escrit:

Show quoted text 9 lines
> > And run-time is shifting to web: > Most desktop applications I know are being moved (or have moved) to the web. > > I believe that soon, the only native apps left will be services and apps that > need hardware access (automation etc.) Well, using Zoom a lot the last months, it taught me that this desktop app solution is much better than any browser based video conferencing system and probably the reason why Zoom apparently won the video conferencing system race.

While I agree that there are still many cases where a desktop app is way better than a web app, I don't think that's the case for video conferencing, I much prefer jitsi-meet (not only because it has the same feature set and it works very well, but because you can self host and not depend on a 3rd party).

Bye

Luca

Reply by email

Wayne Sherman
to Michael Van Canneyt: The problem with the service account is that you must create a JWT Token. FPC does not…
AM Michael Van Canneyt wrote 7 lines
The problem with the service account is that you must create a JWT Token. FPC does not yet have a unit that can generate *and sign* a JWT Token. The good news is I have this weekend a reminder that I received some code that will allow me to complete the JWT Token support in FPC using at least the RSA256 signing algorithm (and some others as well).

Hi Michael. I would like to assist with this if possible.

According to rfc7519:
https://datatracker.ietf.org/doc/html/rfc7519#section-8

The *required* JWT Signature and MAC algorithms are: HS256 (HMAC SHA-256)
none

The *recommended* JWT Signature and MAC algorithms are: RS256 (RSASSA-PKCS1-v1_5 with the SHA-256 hash)
ES256 (ECDSA using the P-256 curve and the SHA-256 hash)

The Google API supports the following signing algorithms: https://cloud.google.com/iot/docs/how-tos/credentials/jwts

JWT RS256 (RSASSA-PKCS1-v1_5 using SHA-256 RFC 7518 sec 3.3). This is expressed as RS256 in the alg field in the JWT header. JWT ES256 (ECDSA using P-256 and SHA-256 RFC 7518 sec 3.4), defined in OpenSSL as the prime256v1 curve. This is expressed as ES256 in the alg field in the JWT header.

So for google api, we need at least RS256 (RSASSA-PKCS1-v1_5 using SHA-256).

Adding signing directly to fpjwt.pp would be the cleanest, but you need to add native crypto code to fpc for that.
https://svn.freepascal.org/svn/fpc/trunk/packages/fcl-web/src/base/fpjwt.pp

There are libraries that have it:
https://github.com/fundamentalslib/fundamentals5
https://github.com/Xor-el/CryptoLib4Pascal

Adding JWT signing to google API units could be done with OpenSSL and it is already a dependency.
OpenSSL can sign and verify a JWT using both RS256 and ES256. Examples: https://stackoverflow.com/questions/58313106/create-rs256-jwt-in-bash https://stackoverflow.com/questions/40559765/how-to-verify-json-web-tokens-with-openssl https://learn.akamai.com/en-us/webhelp/iot/jwt-access-control/GUID-054028C7-1BF8-41A5-BD2E-A3E00F6CA550.html

What do you think?

Reply by email

Wayne Sherman
to Michael Van Canneyt: The problem with the service account is that you must create a JWT Token. FPC does not…
AM Michael Van Canneyt wrote
The problem with the service account is that you must create a JWT Token. FPC does not yet have a unit that can generate *and sign* a JWT Token.

I had a little time this weekend and started looking into this again. I was happy to discover these recent developments:

*Some new hash and elliptic curve crypto files*
https://gitlab.com/freepascal.org/fpc/source/-/tree/main/packages/fcl-hash/src

*Commit which adds support for Java Web Token (JWT) signing* https://gitlab.com/freepascal.org/fpc/source/-/commit/47610d5c651a232a2061a44e2358b147ba65ff1c (fpjwt.pp has new TJWTKey record, TJWTSigner class, and new methods TJWT.Sign and TJWT.ValidateJWT)

*Commit which adds ES256 JWT signing*
https://gitlab.com/freepascal.org/fpc/source/-/commit/5afaeaa3ac813e5682d0da9de7d17454a4771d2c (ES256 signing can be used for Google API service accounts)

Thank you Michael Van Canneyt. :-)

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 20 lines
On Tue, Jun 30, 2020 at 12:47 AM Michael Van Canneyt wrote: > > The problem with the service account is that you must create a JWT Token. > FPC does not yet have a unit that can generate *and sign* a JWT Token. I had a little time this weekend and started looking into this again. I was happy to discover these recent developments: *Some new hash and elliptic curve crypto files* https://gitlab.com/freepascal.org/fpc/source/-/tree/main/packages/fcl-hash/src *Commit which adds support for Java Web Token (JWT) signing* https://gitlab.com/freepascal.org/fpc/source/-/commit/47610d5c651a232a2061a44e2358b147ba65ff1c (fpjwt.pp has new TJWTKey record, TJWTSigner class, and new methods TJWT.Sign and TJWT.ValidateJWT) *Commit which adds ES256 JWT signing* https://gitlab.com/freepascal.org/fpc/source/-/commit/5afaeaa3ac813e5682d0da9de7d17454a4771d2c (ES256 signing can be used for Google API service accounts)

Yes. I would still like to see if we can integrate the OAuth of the google API and the JWT signing more closely, possibly through some extra classes.

Thank you Michael Van Canneyt. :-)

You're welcome. We're slowly improving things.

There is more to come, but I am doing a round-robin on the many areas I am active in, so it may be some weeks before I come back to this... :-)

Michael.

Reply by email

Wayne Sherman
to Michael Van Canneyt
AM Michael Van Canneyt wrote
Yes. I would still like to see if we can integrate the OAuth of the google API and the JWT signing more closely, possibly through some extra classes.

The paint is still wet on the JWT signing code, so I figured there was some more integration to do. I started investigating the "googleapiconv" program, the googleapi examples, and generated API files (which are about 5-years old). I discovered some issues. I will send a report in another email.

There is more to come, but I am doing a round-robin on the many areas I am actively working on in, so it may be some weeks before I come back to this... :-)

Understood. Thanks for your work.

Wayne

Reply by email

Wayne Sherman
to Wayne Sherman: On Tue, Jun 30, 2020 at 12:47 AM Michael Van Canneyt Hi Michael. I would like to assist…
PM Wayne Sherman wrote 8 lines
The Google API supports the following signing algorithms: https://cloud.google.com/iot/docs/how-tos/credentials/jwts JWT RS256 (RSASSA-PKCS1-v1_5 using SHA-256 RFC 7518 sec 3.3). This is expressed as RS256 in the alg field in the JWT header. JWT ES256 (ECDSA using P-256 and SHA-256 RFC 7518 sec 3.4), defined in OpenSSL as the prime256v1 curve. This is expressed as ES256 in the alg field in the JWT header.

My statement above about ES256 does not appear to be correct. After looking over Google's API docs again, I discovered that JWTs signed using ES256 are only supported on a few of their services (?). (e.g. Cloud IoT Core, Identity-Aware Proxy, and Cloud Security Token)

ES256 support was added to the google API python auth library here: https://github.com/googleapis/google-auth-library-python/pull/340

See also:
https://github.com/googleapis/google-auth-library-python/search?q=es256 https://github.com/googleapis/google-api-dotnet-client/search?q=es256

But according to this page:

*Computing the signature*
https://developers.google.com/identity/protocols/oauth2/service-account#httprest

"The signing algorithm in the JWT header must be used when computing the signature. The *only signing algorithm supported* by the Google OAuth 2.0 Authorization Server *is RSA using SHA-256 hashing* algorithm. This is expressed as RS256 in the alg field in the JWT header."

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 31 lines
On Mon, May 10, 2021 at 7:08 PM Wayne Sherman wrote: > The Google API supports the following signing algorithms: > https://cloud.google.com/iot/docs/how-tos/credentials/jwts > > JWT RS256 (RSASSA-PKCS1-v1_5 using SHA-256 RFC 7518 sec 3.3). This > is expressed as RS256 in the alg field in the JWT header. > JWT ES256 (ECDSA using P-256 and SHA-256 RFC 7518 sec 3.4), defined > in OpenSSL as the prime256v1 curve. This is expressed as ES256 in the > alg field in the JWT header. My statement above about ES256 does not appear to be correct. After looking over Google's API docs again, I discovered that JWTs signed using ES256 are only supported on a few of their services (?). (e.g. Cloud IoT Core, Identity-Aware Proxy, and Cloud Security Token) ES256 support was added to the google API python auth library here: https://github.com/googleapis/google-auth-library-python/pull/340 See also: https://github.com/googleapis/google-auth-library-python/search?q=es256 https://github.com/googleapis/google-api-dotnet-client/search?q=es256 But according to this page: *Computing the signature* https://developers.google.com/identity/protocols/oauth2/service-account#httprest "The signing algorithm in the JWT header must be used when computing the signature. The *only signing algorithm supported* by the Google OAuth 2.0 Authorization Server *is RSA using SHA-256 hashing* algorithm. This is expressed as RS256 in the alg field in the JWT header."

RS256 is on my todo list, I have 3 implementations that I can use, but I still need to find time to actually choose and integrate it.

Michael.

Reply by email