CPYFPWN, CPYFMWN, CPYFEWN

Memory Copy Forward-only, writes non-temporal. These instructions perform a memory copy. The prologue, main, and epilogue instructions are expected to be run in succession and to appear consecutively in memory: CPYFPWN, then CPYFMWN, and then CPYFEWN.

CPYFPWN performs some preconditioning of the arguments suitable for using the CPYFMWN instruction, and performs an IMPLEMENTATION DEFINED amount of the memory copy. CPYFMWN performs an IMPLEMENTATION DEFINED amount of the memory copy. CPYFEWN performs the last part of the memory copy.

Note

The inclusion of IMPLEMENTATION DEFINED amounts of memory copy allows some optimization of the size that can be performed.

The memory copy performed by these instructions is in the forward direction only, so the instructions are suitable for a memory copy only where there is no overlap between the source and destination locations, or where the source address is greater than the destination address.

The architecture supports two algorithms for the memory copy: option A and option B. Which algorithm is used is IMPLEMENTATION DEFINED.

Note

Portable software should not assume that the choice of algorithm is constant.

After execution of CPYFPWN, option A (which results in encoding PSTATE.C = 0):

• If Xn<63> == 1, the copy size is saturated to 0x7FFFFFFFFFFFFFFF.
• Xs holds the original Xs + saturated Xn.
• Xd holds the original Xd + saturated Xn.
• Xn holds -1* saturated Xn + an IMPLEMENTATION DEFINED number of bytes copied.
• PSTATE.{N,Z,V} are set to {0,0,0}.

After execution of CPYFPWN, option B (which results in encoding PSTATE.C = 1):

• If Xn<63> == 1, the copy size is saturated to 0x7FFFFFFFFFFFFFFF.
• Xs holds the original Xs + an IMPLEMENTATION DEFINED number of bytes copied.
• Xd holds the original Xd + an IMPLEMENTATION DEFINED number of bytes copied.
• Xn holds the saturated Xn - an IMPLEMENTATION DEFINED number of bytes copied.
• PSTATE.{N,Z,V} are set to {0,0,0}.

For CPYFMWN, option A (encoded by PSTATE.C = 0), the format of the arguments is:

• Xn is treated as a signed 64-bit number and holds -1* the number of bytes remaining to be copied in the memory copy in total.
• Xs holds the lowest address that the copy is copied from -Xn.
• Xd holds the lowest address that the copy is made to -Xn.
• At the end of the instruction, the value of Xn is written back with -1* the number of bytes remaining to be copied in the memory copy in total.

For CPYFMWN, option B (encoded by PSTATE.C = 1), the format of the arguments is:

• Xn holds the number of bytes remaining to be copied in the memory copy in total.
• Xs holds the lowest address that the copy is copied from.
• Xd holds the lowest address that the copy is copied to.
• At the end of the instruction:
  • the value of Xn is written back with the number of bytes remaining to be copied in the memory copy in total.
  • the value of Xs is written back with the lowest address that has not been copied from.
  • the value of Xd is written back with the lowest address that has not been copied to.

For CPYFEWN, option A (encoded by PSTATE.C = 0), the format of the arguments is:

• Xn is treated as a signed 64-bit number and holds -1* the number of bytes remaining to be copied in the memory copy in total.
• Xs holds the lowest address that the copy is copied from -Xn.
• Xd holds the lowest address that the copy is made to -Xn.
• At the end of the instruction, the value of Xn is written back with 0.

For CPYFEWN, option B (encoded by PSTATE.C = 1), the format of the arguments is:

• Xn holds the number of bytes remaining to be copied in the memory copy in total.
• Xs holds the lowest address that the copy is copied from.
• Xd holds the lowest address that the copy is copied to.
• At the end of the instruction:
  • the value of Xn is written back with 0.
  • the value of Xs is written back with the lowest address that has not been copied from.
  • the value of Xd is written back with the lowest address that has not been copied to.

Integer
(FEAT_MOPS)

if !IsFeatureImplemented(FEAT_MOPS) || sz != '00' then UNDEFINED;

CPYParams memcpy;
memcpy.d = UInt(Rd);
memcpy.s = UInt(Rs);
memcpy.n = UInt(Rn);
bits(4) options = op2;
boolean rnontemporal = options<3> == '1';
boolean wnontemporal = options<2> == '1';

case op1 of
    when '00' memcpy.stage = MOPSStage_Prologue;
    when '01' memcpy.stage = MOPSStage_Main;
    when '10' memcpy.stage = MOPSStage_Epilogue;
    otherwise SEE "Memory Copy and Memory Set";

CheckMOPSEnabled();

if (memcpy.s == memcpy.n || memcpy.s == memcpy.d || memcpy.n == memcpy.d || memcpy.d == 31 || memcpy.s == 31 || memcpy.n == 31) then
    Constraint c = ConstrainUnpredictable(Unpredictable_MOPSOVERLAP31);
    assert c IN {Constraint_UNDEF, Constraint_NOP};
    case c of
        when Constraint_UNDEF UNDEFINED;
        when Constraint_NOP   EndOfInstruction();

Assembler Symbols

constant integer N = MaxBlockSizeCopiedBytes();
bits(8*N) readdata;

memcpy.nzcv = PSTATE.<N,Z,C,V>;
memcpy.toaddress = X[memcpy.d, 64];
memcpy.fromaddress = X[memcpy.s, 64];
memcpy.cpysize = SInt(X[memcpy.n, 64]);
memcpy.implements_option_a = CPYFOptionA();

boolean rprivileged = if options<1> == '1' then AArch64.IsUnprivAccessPriv() else PSTATE.EL != EL0;
boolean wprivileged = if options<0> == '1' then AArch64.IsUnprivAccessPriv() else PSTATE.EL != EL0;

AccessDescriptor raccdesc = CreateAccDescMOPS(MemOp_LOAD, rprivileged, rnontemporal);
AccessDescriptor waccdesc = CreateAccDescMOPS(MemOp_STORE, wprivileged, wnontemporal);

if memcpy.stage == MOPSStage_Prologue then
    if memcpy.cpysize<63> == '1' then memcpy.cpysize = 0x7FFFFFFFFFFFFFFF;

    if memcpy.implements_option_a then
        memcpy.nzcv = '0000';
        // Copy in the forward direction offsets the arguments.
        memcpy.toaddress = memcpy.toaddress + memcpy.cpysize;
        memcpy.fromaddress = memcpy.fromaddress + memcpy.cpysize;
        memcpy.cpysize = 0 - memcpy.cpysize;
    else
        memcpy.nzcv = '0010';

memcpy.stagecpysize = MemCpyStageSize(memcpy);

if memcpy.stage != MOPSStage_Prologue then
    CheckMemCpyParams(memcpy, options);

integer copied;
boolean iswrite;
AddressDescriptor memaddrdesc;
PhysMemRetStatus memstatus;
memcpy.forward = TRUE;
boolean fault = FALSE;
integer B;

if memcpy.implements_option_a then
    while memcpy.stagecpysize != 0 && !fault do
        // IMP DEF selection of the block size that is worked on. While many
        // implementations might make this constant, that is not assumed.
        B = CPYSizeChoice(memcpy);
        assert B <= -1 * memcpy.stagecpysize;

        (copied, iswrite, memaddrdesc, memstatus) = MemCpyBytes(memcpy.toaddress + memcpy.cpysize, memcpy.fromaddress + memcpy.cpysize, memcpy.forward, B, raccdesc, waccdesc);
        if copied != B then
            fault = TRUE;
        else
            memcpy.cpysize = memcpy.cpysize + B;
            memcpy.stagecpysize = memcpy.stagecpysize + B;
else
    while memcpy.stagecpysize > 0 && !fault do
        // IMP DEF selection of the block size that is worked on. While many
        // implementations might make this constant, that is not assumed.
        B = CPYSizeChoice(memcpy);
        assert B <= memcpy.stagecpysize;

        (copied, iswrite, memaddrdesc, memstatus) = MemCpyBytes(memcpy.toaddress, memcpy.fromaddress, memcpy.forward, B, raccdesc, waccdesc);
        if copied != B then
            fault = TRUE;
        else
            memcpy.fromaddress = memcpy.fromaddress + B;
            memcpy.toaddress = memcpy.toaddress + B;
            memcpy.cpysize = memcpy.cpysize - B;
            memcpy.stagecpysize = memcpy.stagecpysize - B;

UpdateCpyRegisters(memcpy, fault, copied);

if fault then
    if IsFault(memaddrdesc) then
        AArch64.Abort(memaddrdesc.vaddress, memaddrdesc.fault);

    if IsFault(memstatus) then
        AccessDescriptor accdesc = if iswrite then waccdesc else raccdesc;
        HandleExternalAbort(memstatus, iswrite, memaddrdesc, B, accdesc);

if memcpy.stage == MOPSStage_Prologue then
    PSTATE.<N,Z,C,V> = memcpy.nzcv;