Persistent Storage Manager on nRF51

04 Jul 2015

I have to store some settings and data in persistent storage using my device, but can't do it using mbed USBMSD or LocalFileSystem – https://developer.mbed.org/forum/mbed/topic/16571/.

But nRF51 has the pstorage(more about this API here: https://developer.nordicsemi.com/nRF51_SDK/nRF51_SDK_v8.x.x/doc/8.1.0/s110/html/a00168.html#pstorage_init)

So i've tried to implement it in mbed and have this results:

  • Module initialization successful
  • Registration successful
  • Get Block Identifier successful
  • Store data successfully
  • Load data successful

BUT! I've tried to store 8 bytes, and operations was successful

uint8_t   source_data[8] = {0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07};
retval = pstorage_store(&block_handle, source_data, 8, 0);

then i tried load this 8 bytes, and operations was successful again:

retval = pstorage_load(dest_data, &block_handle, 8, 0);

but: source_data =! dest_data :( my dest_data was like {0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff};

So! my questions are:

  • someone tried implement nRF51 pstorage in mdeb successfuly?
  • on my first pstorage registration i put: param.block_size = 100; param.block_count = 10; for the test, but now i want to change it – how? when i add new params - i have "Failed to register" error. Can i reset the pstorage?
04 Jul 2015

pstorage module on nRF51 works. Please refer to the URIBeacon demo for a working sample. Config parameters are made persistent in the case of URIBeacon using pstorage.

06 Jul 2015

Rohit, thanx for the example, i've looked throw it(https://developer.mbed.org/teams/Bluetooth-Low-Energy/code/BLE_URIBeacon/file/867bf234e47d/nrfConfigParamsPersistence.cpp) and i have a questions:

While you save the data you use base handle without offset:

include the mbed library with this snippet

pstorage_store(&pstorageHandle,
                       reinterpret_cast<uint8_t *>(&persistentParams),
                       sizeof(PersistentParams_t),
                       0 /* offset */);

But when you load the data you use some MAGIC address - what is is for?:

include the mbed library with this snippet

(pstorage_load(reinterpret_cast<uint8_t *>(&persistentParams), &pstorageHandle, sizeof(PersistentParams_t), 0) != NRF_SUCCESS) ||
        (persistentParams.persistenceSignature != PersistentParams_t::MAGIC)

In my example i use definitely the same code to test store and load data:

include the mbed library with this snippet


 uint32_t retval;
    uint8_t   source_data[8] = {0x0a,0x0b,0x0c,0x0d,0x0e,0x0f,0x01,0x02};
    uint8_t   dest_data[8];
    
    pstorage_handle_t handle;
    
    retval = pstorage_init(); 
        if(retval == NRF_SUCCESS)
        {
            serial.printf("Module initialization successful\r\n");
             
            pstorage_module_param_t param;
             
            param.block_size  = 100; 
            param.block_count = 10; 
            
            param.cb = example_cb_handler;
    
            retval = pstorage_register(&param, &handle); //register our pstorage and store store address in handle
            if (retval == NRF_SUCCESS)
            {
                serial.printf("Registration successful.\r\n");
                serial.printf("Module id: %u , block: %u \r\n", handle.module_id, handle.block_id);
                
                
                retval = pstorage_store(&handle, source_data, sizeof(source_data), 0); //store test data in bloc number 0
                if (retval == NRF_SUCCESS)
                {
                   serial.printf("Store successfully requested. Wait for operation result.\r\n");
                   
                   
                    retval = pstorage_load(dest_data, &handle, sizeof(dest_data), 0); //try to load test data
                    if (retval == NRF_SUCCESS)
                    {
                        serial.printf("STORE: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", source_data[0], source_data[1], source_data[2], source_data[3],source_data[4], source_data[5], source_data[6], source_data[7] );
                        serial.printf("LOAD: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", dest_data[0], dest_data[1],dest_data[2],dest_data[3], dest_data[4],dest_data[5],dest_data[6],dest_data[7] );                
                    }
                    else {serial.printf("Failed to load, take corrective action.\r\n"); }                                      
                }
                else {serial.printf("Failed to request store, take corrective action.\r\n");}                                                
            }
            else {serial.printf("Failed to register, take corrective action.\r\n");}
        }
        else {serial.printf("Initialization failed, take corrective action.\r\n"); }

But i've got wrong results:

Module initialization successful
Registration successful.
Module id: 0 , block: 260096 
Store successfully requested. Wait for operation result.
STORE: 0a:0b:0c:0d - 0e:0f:01:02 
LOAD: ff:ff:ff:ff - ff:ff:ff:ff 
Clear successfully requested. Wait for operation result.

07 Jul 2015

You wrote: "But when you load the data you use some MAGIC address"

It is not a magic address. Within nrfConfigParamsPersistence.cpp, PersistentParams_t::MAGIC is the special value which if written into the field 'persistenceSignature' indiacates that data has previously been written to persistent storage successfully.

Please note that the pstorage module is asynchronous. THis means that a call to pstorage_store() will only setup a request for writing into peristent storage. This request will get scheduled and executed in a separate context. The application will receive a notification callback (set during pstorage_register()) upon completion.

refer to https://github.com/mbedmicro/nRF51822/blob/master/source/nordic-sdk/components/drivers_nrf/pstorage/pstorage.h#L219

If you're doing a store following immediately by a load, the pstorage module may not have sufficient time to execute the store before the load. in the case of URIBeacon, the load and store happen at different times, with a sufficiently long interval in between, so this isn't a concern.

If you want to do a load shortly after a store, perhaps you should spin for a period waiting for the notification callback. Please be aware that spinning in a loop after a store() may not be a good idea if done from an interrupt context. The notification callback will be invoked in interrupt context at the same priority as any other callback, so please don't spin after store() if it is called from an interrupt context; if you do, you'll never allow for the notification callback to execute, and the system would spin indefinitely.

Pstorage code comes from Nordic. You should refer to their documentation for further information.

08 Jul 2015

Should anyone else bump into this post, I worked at getting this going from the above code for quite a while with the same error, always getting my load call returning zeros. Turns out I just needed to init the ble and setup the ble wait for event loop to get the soft device to handle the pstorage call, simple thing that I overlooked when putting together a bare bones test app.

11 Jul 2015

i continue my experiments with Persistent Storage and that's my results:

Case #1: I'm trying to store data, wait while my PS will be ready to read this data and try to read it, here is my code in main():

    wait(5);
    serial.printf("Warming up\r\n");
    wait(5);
    
    retval = pstorage_init(); 
        if(retval == NRF_SUCCESS)
        {
            serial.printf("Module initialization successful\r\n");
             
            pstorage_module_param_t param;
             
            param.block_size  = 100; 
            param.block_count = 10; 
            
            param.cb = example_cb_handler;
    
            retval = pstorage_register(&param, &handle); //register our pstorage and store store address in handle
            if (retval == NRF_SUCCESS)
            {
                serial.printf("Registration successful.\r\n");
                serial.printf("Module id: %u , block: %u \r\n", handle.module_id, handle.block_id);
                
                
                retval = pstorage_store(&handle, source_data, sizeof(source_data), 0); //store test data in bloc 0
                if (retval == NRF_SUCCESS)
                {
                   serial.printf("Store successfully requested. Wait for operation result.\r\n");
                   savetime = time(NULL);                                                                  
                }
                else {serial.printf("Failed to request store, take corrective action.\r\n");}                                                
            }
            else {serial.printf("Failed to register, take corrective action.\r\n");}
        }
        else {serial.printf("Initialization failed, take corrective action.\r\n"); }

 while (true) {
                   
        pstorage_access_status_get(&count);
        if (count == 0)
        {
            loadtime = time(NULL);
            serial.printf("No pending operations, time div: %d \r\n", (loadtime-savetime) );          
            retval = pstorage_load(dest_data, &handle, sizeof(dest_data), 0); //try to load test data
            if (retval == NRF_SUCCESS)
            {
                serial.printf("STORE: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", source_data[0], source_data[1], source_data[2], source_data[3],source_data[4], source_data[5], source_data[6], source_data[7] );
                serial.printf("LOAD: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", dest_data[0], dest_data[1],dest_data[2],dest_data[3], dest_data[4],dest_data[5],dest_data[6],dest_data[7] );                
            }
            else {
               serial.printf("Failed to load, take corrective action.\r\n"); 
            } 
            
            break;                           
        }
        else
        {
            serial.printf("Storage access pending, wait!\r\n");
        }    
           
        wait(1);
    }



And here is the result of this code:

Warming up
Module initialization successful
Registration successful.
Module id: 0 , block: 260096 
Store successfully requested. Wait for operation result.
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
Storage access pending, wait!
....

i'm waiting for 10 minutes... Storage access pending, wait!

Case #2: I'm trying to store data and then i add my load code in BLE Connection Callback, so i wait some time and try to connect to my BLE - on connection try to load data:

void connectionCallback(const Gap::ConnectionCallbackParams_t *params)
{
    serial.printf("BLE Connected!\r\n");
    
     pstorage_access_status_get(&count);
        if (count == 0)
        {
            loadtime = time(NULL);
            serial.printf("No pending operations, time div: %d \r\n", (loadtime-savetime) );          
            retval = pstorage_load(dest_data, &handle, sizeof(dest_data), 0); //try to load test data
            if (retval == NRF_SUCCESS)
            {
                serial.printf("STORE: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", source_data[0], source_data[1], source_data[2], source_data[3],source_data[4], source_data[5], source_data[6], source_data[7] );
                serial.printf("LOAD: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", dest_data[0], dest_data[1],dest_data[2],dest_data[3], dest_data[4],dest_data[5],dest_data[6],dest_data[7] );                
            }
            else {
               serial.printf("Failed to load, take corrective action.\r\n"); 
            } 
                                           
        }
        else
        {
            serial.printf("Storage access pending, wait!\r\n");
        }        
}

And here is the result for this case:

Warming up
Module initialization successful
Registration successful.
Module id: 0 , block: 260096 
Store successfully requested. Wait for operation result.
BLE Connected!
Storage access pending, wait!
BLE Disconnected!
BLE Connected!
Storage access pending, wait!
BLE Disconnected!

Ammm, I'm confused, I do not know what to do and why storage is not working :(

11 Jul 2015

Hi,

I just tried to learn the pstorage, it returned success on register, but I failed on the write, the error code is 16, which means "NRF_ERROR_INVALID_ADDR"

The problem resolved it was caused by my memory address which passed by is not 4byte aligned... thanks

11 Jul 2015

I hope Dmitry and others have come across Nordic's documentation for pstorage: https://developer.nordicsemi.com/nRF51_SDK/nRF51_SDK_v8.x.x/doc/8.0.0/s110/html/a00140.html#pstorage_intro

@Dmitry: In case #1 (above), if that code snippet is executing in interrupt context, then it would never yield control to other system interrupts needing attention. Can you please add a printf() within the example_cb_handler() to confirm that pstorage activity is getting scheduled?

@note: From Nordic's pstorage documentation: Note For implementation of interface included in the example, SoftDevice should be enabled and scheduler (if used) should be initialized prior to initializing this module.

@note: for the pstorage module to process pending requests, the Nordic 'scheduler' must be active, or else the following API must get called: pstorage_sys_event_handler(sys_evt);

for the Nordic port of BLE_API, this happens at https://github.com/mbedmicro/nRF51822/blob/master/source/btle/btle.cpp#L48

once BLE::init() is called, pstorage_sys_event_handler() will get called transparently for every system event.

Please realize that we're in the domain of event driven programming. If you're not in the main thread, you're in some interrupt context, and interrupt contexts must yield in order to allow the rest of the system to progress.

13 Jul 2015

Rohit, thanx a lot! After your deep explanation i've found the bug and fix it!

The problem was in registering for system events, i've just add ble.init() before any persistent storage operations and after ble was inited - everything works fine. There is my test program and it's works! Without ble.init there is no event handlers and callback.

#include "mbed.h"
#include "BLE.h"

#include "pstorage.h"
#include "nrf_error.h"

BLEDevice ble;

uint32_t retval, count;
uint8_t   source_data[8] = {0x0a,0x0b,0x0c,0x0d,0x0e,0x0f,0x01,0x02};
uint8_t   dest_data[8];
time_t savetime, loadtime;

pstorage_handle_t handle;
Serial serial(USBTX, USBRX); // tx, rx



static void cb_handler(pstorage_handle_t  * handle,
                               uint8_t              op_code,
                               uint32_t             result,
                               uint8_t            * p_data,
                               uint32_t             data_len)
{
  
  serial.printf("Callback handler successful\r\n");  
  
  switch(op_code)
        {
           case PSTORAGE_CLEAR_OP_CODE:
               if (result == NRF_SUCCESS)
               {
                   serial.printf("Clear operation successful in Callback\r\n");
               }
               else
               {
                   serial.printf("Clear operation failed in Callback\r\n");
               }
               break;

     
          case PSTORAGE_LOAD_OP_CODE:
               if (result == NRF_SUCCESS)
               {
                   serial.printf("Load operation successful in Callback\r\n");                
               }
               else
               {
                   serial.printf("Load operation failed in Callback\r\n");
               }
               break;
     
          case PSTORAGE_STORE_OP_CODE:            
               if (result == NRF_SUCCESS)
               {
                   serial.printf("Store operation successful in Callback\r\n");
               }
               else
               {
                   serial.printf("Store operation failed in Callback\r\n");
               }               
               break;
        }

}

int main(void){

    
    wait(5);
    serial.printf("Warming up\r\n");
    wait(5);
    
    ble.init();
    
    
    retval = pstorage_init(); 
        if(retval == NRF_SUCCESS)
        {
            serial.printf("Module initialization successful\r\n");
             
            pstorage_module_param_t param;
             
            param.block_size  = 100; 
            param.block_count = 10; 
            
            param.cb = cb_handler;
    
            retval = pstorage_register(&param, &handle); //register our pstorage and store store address in handle
            if (retval == NRF_SUCCESS)
            {
                serial.printf("Registration successful.\r\n");
                serial.printf("Module id: %u , block: %u \r\n", handle.module_id, handle.block_id);
                
                
                retval = pstorage_store(&handle, source_data, sizeof(source_data), 0); //store test data in bloc 0
                if (retval == NRF_SUCCESS)
                {
                   serial.printf("Store successfully requested. Wait for operation result.\r\n");
                   
                   pstorage_access_status_get(&count);
                    if (count == 0)
                    {
                       // loadtime = time(NULL);
                        serial.printf("No pending operations \r\n");          
                        retval = pstorage_load(dest_data, &handle, sizeof(dest_data), 0); //try to load test data
                        if (retval == NRF_SUCCESS)
                        {
                            serial.printf("STORE: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", source_data[0], source_data[1], source_data[2], source_data[3],source_data[4], source_data[5], source_data[6], source_data[7] );
                            serial.printf("LOAD: %02x:%02x:%02x:%02x - %02x:%02x:%02x:%02x \r\n", dest_data[0], dest_data[1],dest_data[2],dest_data[3], dest_data[4],dest_data[5],dest_data[6],dest_data[7] );                
                        }
                        else {
                           serial.printf("Failed to load, take corrective action.\r\n"); 
                        } 
                                           
        }
        else
        {
            serial.printf("Storage access pending, wait!\r\n");
            //serial.printf("Time div: %d \r\n", (loadtime-savetime) ); 
        }   
                   
                                                                       
                }
                else {serial.printf("Failed to request store, take corrective action.\r\n");}                                                
            }
            else {serial.printf("Failed to register, take corrective action.\r\n");}
        }
        else {serial.printf("Initialization failed, take corrective action.\r\n"); }
        


     while (true) {
        
        ble.waitForEvent();
      
    }
    
    


}

The result

Warming up
Module initialization successful
Registration successful.
Module id: 0 , block: 260096 
Callback handler successful
Store operation successful in Callback
Store successfully requested. Wait for operation result.
No pending operations 
Callback handler successful
Load operation successful in Callback
STORE: 0a:0b:0c:0d - 0e:0f:01:02 
LOAD: 0a:0b:0c:0d - 0e:0f:01:02 
13 Jul 2015

One more bug i've found, when i changed params:

from

param.block_size  = 100; 
param.block_count = 10; 

to

param.block_size  = sizeof(source_data); 
param.block_count = 12; 

I've got registration error:

Warming up
Module initialization successful
Failed to register, take corrective action.

I checked, and while registration i've got NRF_ERROR_INVALID_PARAM error. How i can reset my previous registration and make it again with new params?

13 Jul 2015
27 Jul 2015

Rohit, thanx for the links, with description of limitations, but for parameters like:

include the mbed library with this snippet

param.block_size  = 100; 
param.block_count = 10; 

i have the same problem: invalid parameters are passed to the API. So, i've tried any other option(90 & 10, 100 & 20 etc) for param.block_size and param.block_count parameters – same problem. Now i can use only 100 for block_size and 10 for block_count how i can reset this?

27 Jul 2015

Dmitry,

I suspect you're hitting https://github.com/ARMmbed/nRF51822/blob/master/source/nordic-sdk/components/drivers_nrf/pstorage/pstorage.c#L889 But I'd just add some printfs to see how far you get in that block of checks. Put on your investigation hat for a while.

29 Nov 2015

Has anyone been using pstorage reliably on mbed with a FOTA update so that the data in pstorage previously written persists in flash after a FOTA update. I have been doing a few experiments with this and despite reading back from the same module and block id in the storage handle after a FOTA update as before it, the data I read back is zeros. According to the callback the load is successful but contains zeros.

Any thoughts?

Allen

30 Nov 2015

After a further bit of debugging the pstorage_init and register calls are both returning NRF_SUCCESS after the FOTA update. However when I try to read from the memory ( same module and block id ) I had written to prior to the FOTA update I get zeros when I do a pstorage_load() ( which also returns NRF_SUCCESS )

Any thoughts what is clearing down this memory?

01 Dec 2015

@allen: FOTA could be erasing all of application memory. I haven't used pstorage across FOTA. It seems you're a pioneer in this.

01 Dec 2015

Hi Rohit,

According to the Nordic SDK ( which I appreciate has certain differences than when the nrf51 is used via the mbed API ) Pstorage is supported between FOTA updates. Obviously useful in some usecases when keys need to be retained between FOTA updates.

Looking at the mbed nrf51822 library I see the compiled hex of the bootloader, s130_nrf51_1.0.0_bootloader.hex , which source is this built from? The headers/api are in nRF51822/source/nordic-sdk/components/libraries/bootloader_dfu but I don't see the source. Within these headers is dfu_types.h and within it I think I see the issue...

#define DFU_APP_DATA_RESERVED 0x0000 /< Size of Application Data that must be preserved between application updates. This value must be a multiple of page size. Page size is 0x400 (1024d) bytes, thus this value must be 0x0000, 0x0400, 0x0800, 0x0C00, 0x1000, etc. */**

I'll have a play around with this and see if it makes a difference.

Allen

01 Dec 2015

Although presumably DFU_APP_DATA_RESERVED is not runtime configurable, in which case we need to rebuild the bootloader, however in mbed it is only supplied as a hex file, s130_nrf51_1.0.0_bootloader.hex

What bootloader source are we using at the moment?

01 Dec 2015

https://github.com/ARMmbed/dfu-bootloader Haven't built that in a while, though. But the sources are up-to-date.

09 Dec 2015

Sorry to re-ignite an old post, but I've been having trouble using the pstorage API, and suspect that recent changes to the library may have broken it.

I can't get my code to compile so I imported the URIBeacon example as is, and updated all libraries. I get the following errors when trying to compile.

Error: Class "GattAttribute" has no member "hasVariableLength" in "nRF51822/source/nRF5xGattServer.cpp", Line: 92, Col: 10 Error: Class "GattAttribute" has no member "hasVariableLength" in "nRF51822/source/nRF5xGattServer.cpp", Line: 127, Col: 14

In my own code I've closely followed the URIBeacon example but keep getting the following errors;

Error: Undefined symbol pstorage_init() (referred from nvParams.cpp.NRF51_DK.o). Error: Undefined symbol pstorage_load(unsigned char*, pstorage_handle_t*, unsigned short, unsigned short) (referred from nvParams.cpp.NRF51_DK.o). Error: Undefined symbol pstorage_store(pstorage_handle_t*, unsigned char*, unsigned short, unsigned short) (referred from nvParams.cpp.NRF51_DK.o). Error: Undefined symbol pstorage_update(pstorage_handle_t*, unsigned char*, unsigned short, unsigned short) (referred from nvParams.cpp.NRF51_DK.o). Error: Undefined symbol pstorage_register(pstorage_module_param_t*, pstorage_handle_t*) (referred from nvParams.cpp.NRF51_DK.o).

Any suggestions would be welcome Andrew

09 Dec 2015

@andrew: GattAttribute::hasVariableLength was introduced recently in BLE_API with commit https://github.com/ARMmbed/ble/commit/87aeff34749d55f6e37c097d6ea86e171608a46b This change is available in the BLE_API available on mbed.org. Perhaps you need to update your BLE_API? Please also update your nRF51822 library.

09 Dec 2015

@Rohit: I've updated everything in the URIBeacon program and the GattAttribute error has been resolved, however I'm now getting the same errors with pstorage that I was getting with my own code.

Error: Undefined symbol pstorage_init() (referred from nrfConfigParamsPersistence.cpp.NRF51_DK.o).

Error: Undefined symbol pstorage_load(unsigned char*, pstorage_handle_t*, unsigned short, unsigned short) (referred from nrfConfigParamsPersistence.cpp.NRF51_DK.o).

Error: Undefined symbol pstorage_store(pstorage_handle_t*, unsigned char*, unsigned short, unsigned short) (referred from nrfConfigParamsPersistence.cpp.NRF51_DK.o).

Error: Undefined symbol pstorage_update(pstorage_handle_t*, unsigned char*, unsigned short, unsigned short) (referred from nrfConfigParamsPersistence.cpp.NRF51_DK.o).

Error: Undefined symbol pstorage_register(pstorage_module_param_t*, pstorage_handle_t*) (referred from nrfConfigParamsPersistence.cpp.NRF51_DK.o).

Platform is nRF51-DK, Default build

10 Dec 2015

@andrew: The missing symbols/functions belong to the file pstorage.c, which comes from nrf51-sdk. We've recently made a few changes to the files inherited from the Nordic SDK, and they now need an extern "C" {} wrapper around the includes. This change was not reflected in the URIBeacon example; causing build failures.

I've updated URIBeacon. It should build without errors now. Thanks for bringing this to our attention.

11 Dec 2015

Thanks, all works well.

07 Feb 2016

The data values I need to store in flash are char arrays. The pstorage functions take uint8_t arrays as input. Could someone give me some guidance on the best way to convert back and forth between char arrays and uint8_t arrays for this purpose?

Any assistance would be helpful!

18 Jul 2016

Dear Rohit sir,

Persistent Storage Manager work on nrf52832 ? if yes then how can implement mbed library?

18 Jul 2016

persistent Storage mangaer can apply on nrf5_iot_sdk_32xx ?

18 Jan 2017

hai

can i get any latest mbed example on persistent storage .................?

thankyou

19 Jan 2017

i have tried this example but i am not getting the stored data . i am getting full zeros. /media/uploads/iqbalpalemad/screenshot_from_2017-01-19_11-53-41.png

24 Apr 2017

Hello, after I use this example of Dmitry Suhamera . My board (nRF51_DK) got trouble. It cannot show serial print. Moreover, when I use basic example BLE_LED and Nordic mobile application to check, the result is Error (0x81) GATT INTERNAL ERROR. I've tried to clear ps but it doesn't back to normal anymore :(. Anyone can help me solve this problem