CPYPRT, CPYMRT, CPYERT

Memory Copy, reads unprivileged. These instructions perform a memory copy. The prologue, main, and epilogue instructions are expected to be run in succession and to appear consecutively in memory: CPYPRT, then CPYMRT, and then CPYERT.

CPYPRT performs some preconditioning of the arguments suitable for using the CPYMRT instruction, and performs an IMPLEMENTATION DEFINED amount of the memory copy. CPYMRT performs an IMPLEMENTATION DEFINED amount of the memory copy. CPYERT performs the last part of the memory copy.

Note

The inclusion of IMPLEMENTATION DEFINED amounts of memory copy allows some optimization of the size that can be performed.

For CPYPRT, the following saturation logic is applied:

If Xn<63:55> != 000000000, the copy size Xn is saturated to 0x007FFFFFFFFFFFFF.

After that saturation logic is applied, the direction of the memory copy is based on the following algorithm:

If (Xs > Xd) && (Xd + saturated Xn) > Xs, then direction = forward

Elsif (Xs < Xd) && (Xs + saturated Xn) > Xd, then direction = backward

Else direction = IMPLEMENTATION DEFINED choice between forward and backward.

The architecture supports two algorithms for the memory copy: option A and option B. Which algorithm is used is IMPLEMENTATION DEFINED.

Note

Portable software should not assume that the choice of algorithm is constant.

After execution of CPYPRT, option A (which results in encoding PSTATE.C = 0):

• PSTATE.{N,Z,V} are set to {0,0,0}.
• If the copy is in the forward direction, then:
  • Xs holds the original Xs + saturated Xn.
  • Xd holds the original Xd + saturated Xn.
  • Xn holds -1* saturated Xn + an IMPLEMENTATION DEFINED number of bytes copied.
• If the copy is in the backward direction, then:
  • Xs and Xd are unchanged.
  • Xn holds the saturated value of Xn - an IMPLEMENTATION DEFINED number of bytes copied.

After execution of CPYPRT, option B (which results in encoding PSTATE.C = 1):

• If the copy is in the forward direction, then:
  • Xs holds the original Xs + an IMPLEMENTATION DEFINED number of bytes copied.
  • Xd holds the original Xd + an IMPLEMENTATION DEFINED number of bytes copied.
  • Xn holds the saturated Xn - an IMPLEMENTATION DEFINED number of bytes copied.
  • PSTATE.{N,Z,V} are set to {0,0,0}.
• If the copy is in the backward direction, then:
  • Xs holds the original Xs + saturated Xn - an IMPLEMENTATION DEFINED number of bytes copied.
  • Xd holds the original Xd + saturated Xn - an IMPLEMENTATION DEFINED number of bytes copied.
  • Xn holds the saturated Xn - an IMPLEMENTATION DEFINED number of bytes copied.
  • PSTATE.{N,Z,V} are set to {1,0,0}.

For CPYMRT, option A (encoded by PSTATE.C = 0), the format of the arguments is:

• Xn is treated as a signed 64-bit number.
• If the copy is in the forward direction (Xn is a negative number), then:
  • Xn holds -1* the number of bytes remaining to be copied in the memory copy in total.
  • Xs holds the lowest address that the copy is copied from -Xn.
  • Xd holds the lowest address that the copy is made to -Xn.
  • At the end of the instruction, the value of Xn is written back with -1* the number of bytes remaining to be copied in the memory copy in total.
• If the copy is in the backward direction (Xn is a positive number), then:
  • Xn holds the number of bytes remaining to be copied in the memory copy in total.
  • Xs holds the highest address that the copy is copied from -Xn+1.
  • Xd holds the highest address that the copy is copied to -Xn+1.
  • At the end of the instruction, the value of Xn is written back with the number of bytes remaining to be copied in the memory copy in total.

For CPYMRT, option B (encoded by PSTATE.C = 1), the format of the arguments is:

• Xn holds the number of bytes to be copied in the memory copy in total.
• If the copy is in the forward direction (PSTATE.N == 0), then:
  • Xs holds the lowest address that the copy is copied from.
  • Xd holds the lowest address that the copy is copied to.
  • At the end of the instruction:
    • the value of Xn is written back with the number of bytes remaining to be copied in the memory copy in total.
    • the value of Xs is written back with the lowest address that has not been copied from.
    • the value of Xd is written back with the lowest address that has not been copied to.
• If the copy is in the backward direction (PSTATE.N == 1), then:
  • Xs holds the highest address that the copy is copied from +1.
  • Xd holds the highest address that the copy is copied to +1.
  • At the end of the instruction:
    • the value of Xn is written back with the number of bytes remaining to be copied in the memory copy in total.
    • the value of Xs is written back with the highest address that has not been copied from +1.
    • the value of Xd is written back with the highest address that has not been copied to +1.

For CPYERT, option A (encoded by PSTATE.C = 0), the format of the arguments is:

• Xn is treated as a signed 64-bit number.
• If the copy is in the forward direction (Xn is a negative number), then:
  • Xn holds -1* the number of bytes remaining to be copied in the memory copy in total.
  • Xs holds the lowest address that the copy is copied from -Xn.
  • Xd holds the lowest address that the copy is made to -Xn.
  • At the end of the instruction, the value of Xn is written back with 0.
• If the copy is in the backward direction (Xn is a positive number), then:
  • Xn holds the number of bytes remaining to be copied in the memory copy in total.
  • Xs holds the highest address that the copy is copied from -Xn+1.
  • Xd holds the highest address that the copy is copied to -Xn+1.
  • At the end of the instruction, the value of Xn is written back with 0.

For CPYERT, option B (encoded by PSTATE.C = 1), the format of the arguments is:

• Xn holds the number of bytes to be copied in the memory copy in total.
• If the copy is in the forward direction (PSTATE.N == 0), then:
  • Xs holds the lowest address that the copy is copied from.
  • Xd holds the lowest address that the copy is copied to.
  • At the end of the instruction:
    • the value of Xn is written back with 0.
    • the value of Xs is written back with the lowest address that has not been copied from.
    • the value of Xd is written back with the lowest address that has not been copied to.
• If the copy is in the backward direction (PSTATE.N == 1), then:
  • Xs holds the highest address that the copy is copied from +1.
  • Xd holds the highest address that the copy is copied to +1.
  • At the end of the instruction:
    • the value of Xn is written back with 0.
    • the value of Xs is written back with the highest address that has not been copied from +1.
    • the value of Xd is written back with the highest address that has not been copied to +1.

Integer
(FEAT_MOPS)

if !IsFeatureImplemented(FEAT_MOPS) || sz != '00' then UNDEFINED;

CPYParams memcpy;
memcpy.d = UInt(Rd);
memcpy.s = UInt(Rs);
memcpy.n = UInt(Rn);
bits(4) options = op2;
boolean rnontemporal = options<3> == '1';
boolean wnontemporal = options<2> == '1';

case op1 of
    when '00' memcpy.stage = MOPSStage_Prologue;
    when '01' memcpy.stage = MOPSStage_Main;
    when '10' memcpy.stage = MOPSStage_Epilogue;
    otherwise SEE "Memory Copy and Memory Set";

CheckMOPSEnabled();

if (memcpy.s == memcpy.n || memcpy.s == memcpy.d || memcpy.n == memcpy.d || memcpy.d == 31 || memcpy.s == 31 || memcpy.n == 31) then
    Constraint c = ConstrainUnpredictable(Unpredictable_MOPSOVERLAP31);
    assert c IN {Constraint_UNDEF, Constraint_NOP};
    case c of
        when Constraint_UNDEF UNDEFINED;
        when Constraint_NOP   EndOfInstruction();

Assembler Symbols

constant integer N = MaxBlockSizeCopiedBytes();
bits(8*N) readdata;

memcpy.nzcv = PSTATE.<N,Z,C,V>;
memcpy.toaddress = X[memcpy.d, 64];
memcpy.fromaddress = X[memcpy.s, 64];

if memcpy.stage == MOPSStage_Prologue then
    memcpy.cpysize = UInt(X[memcpy.n, 64]);
else
    memcpy.cpysize = SInt(X[memcpy.n, 64]);

memcpy.implements_option_a = CPYOptionA();

boolean rprivileged = if options<1> == '1' then AArch64.IsUnprivAccessPriv() else PSTATE.EL != EL0;
boolean wprivileged = if options<0> == '1' then AArch64.IsUnprivAccessPriv() else PSTATE.EL != EL0;

AccessDescriptor raccdesc = CreateAccDescMOPS(MemOp_LOAD, rprivileged, rnontemporal);
AccessDescriptor waccdesc = CreateAccDescMOPS(MemOp_STORE, wprivileged, wnontemporal);

if memcpy.stage == MOPSStage_Prologue then
    if memcpy.cpysize > 0x007FFFFFFFFFFFFF then
        memcpy.cpysize = 0x007FFFFFFFFFFFFF;

    memcpy.forward = IsMemCpyForward(memcpy);

    if memcpy.implements_option_a then
        memcpy.nzcv = '0000';
        if memcpy.forward then
            // Copy in the forward direction offsets the arguments.
            memcpy.toaddress = memcpy.toaddress + memcpy.cpysize;
            memcpy.fromaddress = memcpy.fromaddress + memcpy.cpysize;
            memcpy.cpysize = 0 - memcpy.cpysize;
    else
        if !memcpy.forward then
            // Copy in the reverse direction offsets the arguments.
            memcpy.toaddress = memcpy.toaddress + memcpy.cpysize;
            memcpy.fromaddress = memcpy.fromaddress + memcpy.cpysize;
            memcpy.nzcv = '1010';
        else
            memcpy.nzcv = '0010';

memcpy.stagecpysize = MemCpyStageSize(memcpy);

if memcpy.stage != MOPSStage_Prologue then
    memcpy.forward = memcpy.cpysize < 0 || (!memcpy.implements_option_a && memcpy.nzcv<3> == '0');
    CheckMemCpyParams(memcpy, options);

integer copied;
boolean iswrite;
AddressDescriptor memaddrdesc;
PhysMemRetStatus memstatus;
boolean fault = FALSE;
integer B;

if memcpy.implements_option_a then
    while memcpy.stagecpysize != 0 && !fault do
        // IMP DEF selection of the block size that is worked on. While many
        // implementations might make this constant, that is not assumed.
        B = CPYSizeChoice(memcpy);

        if memcpy.forward then
            assert B <= -1 * memcpy.stagecpysize;
            (copied, iswrite, memaddrdesc, memstatus) = MemCpyBytes(memcpy.toaddress + memcpy.cpysize, memcpy.fromaddress + memcpy.cpysize, memcpy.forward, B, raccdesc, waccdesc);
            if copied != B then
                fault = TRUE;
            else
                memcpy.cpysize = memcpy.cpysize + B;
                memcpy.stagecpysize = memcpy.stagecpysize + B;

        else
            assert B <= memcpy.stagecpysize;
            memcpy.cpysize = memcpy.cpysize - B;
            memcpy.stagecpysize = memcpy.stagecpysize - B;

            (copied, iswrite, memaddrdesc, memstatus) = MemCpyBytes(memcpy.toaddress + memcpy.cpysize, memcpy.fromaddress + memcpy.cpysize, memcpy.forward, B, raccdesc, waccdesc);
            if copied != B then
                fault = TRUE;
                memcpy.cpysize = memcpy.cpysize + B;
                memcpy.stagecpysize = memcpy.stagecpysize + B;

else
    while memcpy.stagecpysize > 0 && !fault do
        // IMP DEF selection of the block size that is worked on. While many
        // implementations might make this constant, that is not assumed.
        B = CPYSizeChoice(memcpy);
        assert B <= memcpy.stagecpysize;

        if memcpy.forward then
            (copied, iswrite, memaddrdesc, memstatus) = MemCpyBytes(memcpy.toaddress, memcpy.fromaddress, memcpy.forward, B, raccdesc, waccdesc);
            if copied != B then
                fault = TRUE;
            else
                memcpy.fromaddress = memcpy.fromaddress + B;
                memcpy.toaddress = memcpy.toaddress + B;
        else
            (copied, iswrite, memaddrdesc, memstatus) = MemCpyBytes(memcpy.toaddress - B, memcpy.fromaddress - B, memcpy.forward, B, raccdesc, waccdesc);

            if copied != B then
                fault = TRUE;
            else
                memcpy.fromaddress = memcpy.fromaddress - B;
                memcpy.toaddress = memcpy.toaddress - B;

        if !fault then
            memcpy.cpysize = memcpy.cpysize - B;
            memcpy.stagecpysize = memcpy.stagecpysize - B;

UpdateCpyRegisters(memcpy, fault, copied);

if fault then
    if IsFault(memaddrdesc) then
        AArch64.Abort(memaddrdesc.vaddress, memaddrdesc.fault);
    else
        AccessDescriptor accdesc = if iswrite then waccdesc else raccdesc;
        HandleExternalAbort(memstatus, iswrite, memaddrdesc, B, accdesc);

if memcpy.stage == MOPSStage_Prologue then
    PSTATE.<N,Z,C,V> = memcpy.nzcv;