[Linux-Xtensa] RAM load/store reorder

[Max Filippov]

On Thu, Nov 28, 2013 at 5:13 PM, Baruch Siach <baruch at tkos.co.il> wrote:
> Hi Max,
>
> On Thu, Nov 28, 2013 at 03:17:12PM +0400, Max Filippov wrote:
>> On Wed, Nov 27, 2013 at 5:09 PM, Baruch Siach <baruch at tkos.co.il> wrote:
>> > Hi linux-xtensa list,
>> >
>> > I have ported U-Boot v2013.07 (based on
>> > https://github.com/czankel/xtensa-uboot) to our platform running on the KC705
>> > board. Everything seems to work fine except the following little problem. For
>> > this code from lib/libfdt/fdt.c:
>> >
>> > int fdt_next_subnode(const void *fdt, int offset)
>> > {
>> >         int depth = 1;
>> >
>> >         /*
>> >          * With respect to the parent, the depth of the next subnode will be
>> >          * the same as the last.
>> >          */
>> >         do {
>> >                 offset = fdt_next_node(fdt, offset, &depth);
>> >                 if (offset < 0 || depth < 1)
>> >                         return -FDT_ERR_NOTFOUND;
>> >         } while (depth > 1);
>> >
>> >         return offset;
>> > }
>> >
>> > the following code is generated (first few instructions):
>> >
>> > 05f3d408 <fdt_next_subnode>:
>> >  5f3d408:       008136                  entry   a1, 64
>> >  5f3d40b:       03bd                    mov.n   a11, a3
>> >  5f3d40d:       cd8c31                  l32r    a3, 5f30a40
>> >  5f3d410:       180c                    movi.n  a8, 1
>> >  5f3d412:       0189                    s32i.n  a8, a1, 0
>> >  5f3d414:       4139                    s32i.n  a3, a1, 16
>> >  5f3d416:       4138                    l32i.n  a3, a1, 16
>> >  5f3d418:       02ad                    mov.n   a10, a2
>> >  5f3d41a:       20c110                  or      a12, a1, a1
>> >  5f3d41d:       0003e0                  callx8  a3
>> >
>> > Running this code under a debugger with hardware breakpoints at 0x5f3d410 and
>> > 0x5f3d41d indicates that the values at a3 and *(a1+16) got swapped. The
>> > following pseudo code illustrates the situation:
>> >
>> >         tmp      <- a3
>> >         a3       <- *(a1+16)
>> >         *(a1+16) <- tmp
>>
>> What kind of memory a1+16 points to? Is it DTCM or external memory?
>
> a1+16 is in external memory.
>
>> If that's external what are the caching attributes? Does changing memory
>> type or caching attributes affect the issue?
>
> I haven't found any code changing the TLB in U-Boot. So I guess that the reset
> default bypass access mode applies. What other access mode would you suggest?

I'd enable write-through caching for the stack, which would make sure that
written data have reached the memory, while reads from the recently written
addresses are served from the cache. I guess that if read was actually
reordered with the write the change would not affect that and your issue will
still be observable. If OTOH the issue is in the external memory connection
the caching would probably work around it.

-- 
Thanks.
-- Max