qpid-proton mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "German Shepherd (PrE) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PROTON-1021) ProtonE - the Deeply Embedded ProtonC variant
Date Fri, 16 Oct 2015 13:01:05 GMT

     [ https://issues.apache.org/jira/browse/PROTON-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

German Shepherd (PrE) updated PROTON-1021:
------------------------------------------
    Description: 
Let me introduce a project I've started on 2013, when I've been tasked with a solution on
how to connect an deeply embedded Wi-Fi enabled device to MS Azure Cloud (the plan is to connect
millions of such devices).

The obvious solution was to go with ServiceBus and AMQP over TLS. Since the embedded stuff
is always written in plain C, the ProtonC is choice was obvious.

The Proton-C typically requires a 230kB of RAM by default (most of it dynamically allocated),
which is way too much of what is physically available in any embedded CPU of the general class
(those typically end up with RAM sizes in 192kB area) - so there is no way on how to directly
use the ProtonC on such CPUs.

*Solution?*
The outcome of my work is the *Proton-E*, based off the Proton-C library, squeezed to run
on 32kB of total RAM usage (without sacrificing any functionality) and with a TLS on side
taking addition approx. 11kB.
This allows you to fit the AMQPS into any connected embedded CPU with 128kB of RAM (or more)
- which most of the commonly available CPUs on market provide (for a great price).

The "embedded" modifications are backwards applicable to the ProtonC, and this is what I would
like to introduce through this JIRA Ticket.

*Note*
This work was picked up by Microsoft in mid-2014, and I have personally cooperated with them;
but I have not seen any result being provided to community, as it seems the whole effort on
their side vanished (and if not, they stopped the cooperation - so they won't have my latest
sources).

Never mind, as I have the most actual source code (as I've never stopped the development,
and I do follow every commit the ProtonC community does), none is lost.

At this moment, before the ProtonE source gets out, let me use this ticket to provide an assurance,
that indeed the ProtonE (resp. modified ProtonC) can be used on the constrained system, and,
most importantly, it is production ready for Azure Cloud.

*ProtonE Features*
* Super-Low RAM footprint: total 32kB of RAM usage, for a AMQP message with 1.5kB of payload
* Azure Cloud: Service Bus / eventHub - both verified (IoT Hub - very close future, will definitely
work - my colleagues cooperate with MS on this)
* Azure Cloud: SASL PLAIN over TLS (Active Directory) - verified
* Azure Cloud: SAS tokens (CBS) - verified
* No modifications to ProtonC API what-so-ever
* TCP stack, and RTOS, independent

*The ProtonE has been made to run and personally tested, by myself, with the following setup:
*
* CPUs: 32-bit ARM - Cortex M3 (STM32, SAM3), Cortex M4 (Atmel SAM4), Cortex M7 (Atmel SAM
S70/V71), I also run this on TI Sitara with embedded RTOS
* There is no reason why this would not work on any other CPUs (all is just a plain-C code)
* RTOS: FreeRTOS/SafeRTOS; but primarily: expressLogic ThreadX
* TCP/IP stack: expressLogic NetX & NetX-Duo TCP/IP
* Comms: Broadcom WICED Wi-Fi solution
* TLS via Mocana NanoSSL (v6.0 or newer)
* TLS via MatrixSSL (3.6.1 or newer)

*SSL/TLS*
This is quite a huge memory hog, but should you understand it deeply, you won't have trouble
managing this.
Some numbers I've got:
* Handshake establishment: RAM usage peak is 30kB for TLS itself (handshake: RSA)
* After the Handshake is done and the TLS tunnel is open: the TLS itself requires 11kB of
RAM (tunnel: AES)
I'm looking forward to have Microsoft to switch the Azure solely to ECC for us (now they have
RSA and ECC, and the server-side cert is solely RSA). Then the handshake number will be even
much more positive.

*Final words*
Once you realize that the AMQPS is the only possible scalable future for the IoT (... hate
this buzzword, BTW.), where the 95% of devices will be deeply embedded ones, then it is obvious
what role the ProtonE will play.

Best regards from CZ,
 Adam N

  was:
Let me introduce a project I've started on 2013, when I've been tasked with a solution on
how to connect an deeply embedded Wi-Fi enabled device to MS Azure Cloud (the plan is to connect
millions of such devices).

The obvious solution was to go with ServiceBus and AMQP over TLS. Since the embedded stuff
is always written in plain C, the ProtonC is choice was obvious.

The Proton-C typically requires a 230kB of RAM by default (most of it dynamically allocated),
which is way too much of what is physically available in any embedded CPU of the general class
(those typically end up with RAM sizes in 192kB area) - so there is no way on how to directly
use the ProtonC on such CPUs.

*Solution?*
The outcome of my work is the *Proton-E*, based off the Proton-C library, squeezed to run
on 32kB of total RAM usage (without sacrificing any functionality) and with a TLS on side
taking addition approx. 11kB.
This allows you to fit the AMQPS into any connected embedded CPU with 128kB of RAM (or more)
- which most of the commonly available CPUs on market provide (for a great price).

The "embedded" modifications are backwards applicable to the ProtonC, and this is what I would
like to introduce through this JIRA Ticket.

*Note*
This work was picked up by Microsoft in mid-2014, and I have personally cooperated with them;
but I have not seen any result being provided to community, as it seems the whole effort on
their side vanished (and if not, they stopped the cooperation - so they won't have my latest
sources).

Never mind, as I have the most actual source code (as I've never stopped the development,
and I do follow every commit the ProtonC community does), none is lost.

At this moment, before the ProtonE source gets out, let me use this ticket to provide an assurance,
that indeed the ProtonE (resp. modified ProtonC) can be used on the constrained system, and,
most importantly, it is production ready for Azure Cloud.

*ProtonE Features*
* Super-Low RAM footprint: total 32kB of RAM usage, for a AMQP message with 1.5kB of payload
* Azure Cloud: Service Bus / eventHub / IoT Hub - verified
* Azure Cloud: SASL PLAIN over TLS (Active Directory) - verified
* Azure Cloud: SAS tokens (CBS) - verified
* No modifications to ProtonC API what-so-ever
* TCP stack, and RTOS, independent

* The ProtonE has been made to run and personally tested, by myself, with the following setup:
*
* CPUs: 32-bit ARM - Cortex M3 (STM32, SAM3), Cortex M4 (Atmel SAM4), Cortex M7 (Atmel SAM
S70/V71), I also run this on TI Sitara with embedded RTOS
* RTOS: FreeRTOS/SafeRTOS; but primarily: expressLogic ThreadX
* TCP/IP stack: expressLogic NetX & NetX-Duo TCP/IP
* Comms: Broadcom WICED Wi-Fi solution
* TLS via Mocana NanoSSL (v6.0 or newer)
* TLS via MatrixSSL (3.6.1 or newer)

*SSL/TLS*
This is quite a huge memory hog, but should you understand it deeply, you won't have trouble
managing this.
Some numbers I've got:
* Handshake establishment: RAM usage peak is 30kB for TLS itself (handshake: RSA)
* After the Handshake is done and the TLS tunnel is open: the TLS itself requires 11kB of
RAM (tunnel: AES)
I'm looking forward to have Microsoft to switch the Azure solely to ECC for us (now they have
RSA and ECC, and the server-side cert is solely RSA). Then the handshake number will be even
much more positive.

*Final words*
Once you realize that the AMQPS is the only possible scalable future for the IoT (... hate
this buzzword, BTW.), where the 95% of devices will be deeply embedded ones, then it is obvious
what role the ProtonE will play.

Best regards from CZ,
 Adam N


> ProtonE - the Deeply Embedded ProtonC variant
> ---------------------------------------------
>
>                 Key: PROTON-1021
>                 URL: https://issues.apache.org/jira/browse/PROTON-1021
>             Project: Qpid Proton
>          Issue Type: Improvement
>          Components: proton-c
>         Environment: Embedded C - IAR or GCC.
>            Reporter: German Shepherd (PrE)
>            Priority: Minor
>              Labels: features
>
> Let me introduce a project I've started on 2013, when I've been tasked with a solution
on how to connect an deeply embedded Wi-Fi enabled device to MS Azure Cloud (the plan is to
connect millions of such devices).
> The obvious solution was to go with ServiceBus and AMQP over TLS. Since the embedded
stuff is always written in plain C, the ProtonC is choice was obvious.
> The Proton-C typically requires a 230kB of RAM by default (most of it dynamically allocated),
which is way too much of what is physically available in any embedded CPU of the general class
(those typically end up with RAM sizes in 192kB area) - so there is no way on how to directly
use the ProtonC on such CPUs.
> *Solution?*
> The outcome of my work is the *Proton-E*, based off the Proton-C library, squeezed to
run on 32kB of total RAM usage (without sacrificing any functionality) and with a TLS on side
taking addition approx. 11kB.
> This allows you to fit the AMQPS into any connected embedded CPU with 128kB of RAM (or
more) - which most of the commonly available CPUs on market provide (for a great price).
> The "embedded" modifications are backwards applicable to the ProtonC, and this is what
I would like to introduce through this JIRA Ticket.
> *Note*
> This work was picked up by Microsoft in mid-2014, and I have personally cooperated with
them; but I have not seen any result being provided to community, as it seems the whole effort
on their side vanished (and if not, they stopped the cooperation - so they won't have my latest
sources).
> Never mind, as I have the most actual source code (as I've never stopped the development,
and I do follow every commit the ProtonC community does), none is lost.
> At this moment, before the ProtonE source gets out, let me use this ticket to provide
an assurance, that indeed the ProtonE (resp. modified ProtonC) can be used on the constrained
system, and, most importantly, it is production ready for Azure Cloud.
> *ProtonE Features*
> * Super-Low RAM footprint: total 32kB of RAM usage, for a AMQP message with 1.5kB of
payload
> * Azure Cloud: Service Bus / eventHub - both verified (IoT Hub - very close future, will
definitely work - my colleagues cooperate with MS on this)
> * Azure Cloud: SASL PLAIN over TLS (Active Directory) - verified
> * Azure Cloud: SAS tokens (CBS) - verified
> * No modifications to ProtonC API what-so-ever
> * TCP stack, and RTOS, independent
> *The ProtonE has been made to run and personally tested, by myself, with the following
setup: *
> * CPUs: 32-bit ARM - Cortex M3 (STM32, SAM3), Cortex M4 (Atmel SAM4), Cortex M7 (Atmel
SAM S70/V71), I also run this on TI Sitara with embedded RTOS
> * There is no reason why this would not work on any other CPUs (all is just a plain-C
code)
> * RTOS: FreeRTOS/SafeRTOS; but primarily: expressLogic ThreadX
> * TCP/IP stack: expressLogic NetX & NetX-Duo TCP/IP
> * Comms: Broadcom WICED Wi-Fi solution
> * TLS via Mocana NanoSSL (v6.0 or newer)
> * TLS via MatrixSSL (3.6.1 or newer)
> *SSL/TLS*
> This is quite a huge memory hog, but should you understand it deeply, you won't have
trouble managing this.
> Some numbers I've got:
> * Handshake establishment: RAM usage peak is 30kB for TLS itself (handshake: RSA)
> * After the Handshake is done and the TLS tunnel is open: the TLS itself requires 11kB
of RAM (tunnel: AES)
> I'm looking forward to have Microsoft to switch the Azure solely to ECC for us (now they
have RSA and ECC, and the server-side cert is solely RSA). Then the handshake number will
be even much more positive.
> *Final words*
> Once you realize that the AMQPS is the only possible scalable future for the IoT (...
hate this buzzword, BTW.), where the 95% of devices will be deeply embedded ones, then it
is obvious what role the ProtonE will play.
> Best regards from CZ,
>  Adam N



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message