| 1 | Hexagon is Qualcomm's very long instruction word (VLIW) digital signal |
| 2 | processor(DSP). We also support Hexagon Vector eXtensions (HVX). HVX |
| 3 | is a wide vector coprocessor designed for high performance computer vision, |
| 4 | image processing, machine learning, and other workloads. |
| 5 | |
| 6 | The following versions of the Hexagon core are supported |
| 7 | Scalar core: v73 |
| 8 | https://developer.qualcomm.com/downloads/qualcomm-hexagon-v73-programmers-reference-manual-rev-aa |
| 9 | HVX extension: v73 |
| 10 | https://developer.qualcomm.com/downloads/qualcomm-hexagon-v73-hvx-programmers-reference-manual-rev-aa |
| 11 | |
| 12 | We presented an overview of the project at the 2019 KVM Forum. |
| 13 | https://kvmforum2019.sched.com/event/Tmwc/qemu-hexagon-automatic-translation-of-the-isa-manual-pseudcode-to-tiny-code-instructions-of-a-vliw-architecture-niccolo-izzo-revng-taylor-simpson-qualcomm-innovation-center |
| 14 | |
| 15 | *** Tour of the code *** |
| 16 | |
| 17 | The qemu-hexagon implementation is a combination of qemu and the Hexagon |
| 18 | architecture library (aka archlib). The three primary directories with |
| 19 | Hexagon-specific code are |
| 20 | |
| 21 | qemu/target/hexagon |
| 22 | This has all the instruction and packet semantics |
| 23 | qemu/target/hexagon/imported |
| 24 | These files are imported with very little modification from archlib |
| 25 | *.idef Instruction semantics definition |
| 26 | macros.def Mapping of macros to instruction attributes |
| 27 | encode*.def Encoding patterns for each instruction |
| 28 | iclass.def Instruction class definitions used to determine |
| 29 | legal VLIW slots for each instruction |
| 30 | qemu/target/hexagon/idef-parser |
| 31 | Parser that, given the high-level definitions of an instruction, |
| 32 | produces a C function generating equivalent tiny code instructions. |
| 33 | See README.rst. |
| 34 | qemu/linux-user/hexagon |
| 35 | Helpers for loading the ELF file and making Linux system calls, |
| 36 | signals, etc |
| 37 | |
| 38 | We start with scripts that generate a bunch of include files. This |
| 39 | is a two step process. The first step is to use the C preprocessor to expand |
| 40 | macros inside the architecture definition files. This is done in |
| 41 | target/hexagon/gen_semantics.c. This step produces |
| 42 | <BUILD_DIR>/target/hexagon/semantics_generated.pyinc. |
| 43 | That file is consumed by the following python scripts to produce the indicated |
| 44 | header files in <BUILD_DIR>/target/hexagon |
| 45 | gen_opcodes_def.py -> opcodes_def_generated.h.inc |
| 46 | gen_printinsn.py -> printinsn_generated.h.inc |
| 47 | gen_op_attribs.py -> op_attribs_generated.h.inc |
| 48 | gen_helper_protos.py -> helper_protos_generated.h.inc |
| 49 | gen_tcg_funcs.py -> tcg_funcs_generated.c.inc |
| 50 | gen_helper_funcs.py -> helper_funcs_generated.c.inc |
| 51 | gen_idef_parser_funcs.py -> idef_parser_input.h |
| 52 | gen_analyze_funcs.py -> analyze_funcs_generated.c.inc |
| 53 | |
| 54 | Qemu helper functions have 3 parts |
| 55 | DEF_HELPER declaration indicates the signature of the helper |
| 56 | gen_helper_<NAME> will generate a TCG call to the helper function |
| 57 | The helper implementation |
| 58 | |
| 59 | Here's an example of the A2_add instruction. |
| 60 | Instruction tag A2_add |
| 61 | Assembly syntax "Rd32=add(Rs32,Rt32)" |
| 62 | Instruction semantics "{ RdV=RsV+RtV;}" |
| 63 | |
| 64 | By convention, the operands are identified by letter |
| 65 | RdV is the destination register |
| 66 | RsV, RtV are source registers |
| 67 | |
| 68 | The generator uses the operand naming conventions (see large comment in |
| 69 | hex_common.py) to determine the signature of the helper function. Here are the |
| 70 | results for A2_add |
| 71 | |
| 72 | helper_protos_generated.h.inc |
| 73 | DEF_HELPER_3(A2_add, s32, env, s32, s32) |
| 74 | |
| 75 | tcg_funcs_generated.c.inc |
| 76 | static void generate_A2_add( |
| 77 | CPUHexagonState *env, |
| 78 | DisasContext *ctx, |
| 79 | Insn *insn, |
| 80 | Packet *pkt) |
| 81 | { |
| 82 | Insn *insn G_GNUC_UNUSED = ctx->insn; |
| 83 | const int RdN = insn->regno[0]; |
| 84 | TCGv RdV = get_result_gpr(ctx, RdN); |
| 85 | const int RsN = insn->regno[1]; |
| 86 | TCGv RsV = hex_gpr[RsN]; |
| 87 | const int RtN = insn->regno[2]; |
| 88 | TCGv RtV = hex_gpr[RtN]; |
| 89 | gen_helper_A2_add(RdV, tcg_env, RsV, RtV); |
| 90 | } |
| 91 | |
| 92 | helper_funcs_generated.c.inc |
| 93 | int32_t HELPER(A2_add)(CPUHexagonState *env, int32_t RsV, int32_t RtV) |
| 94 | { |
| 95 | uint32_t slot __attribute__((unused)) = 4; |
| 96 | int32_t RdV = 0; |
| 97 | { RdV=RsV+RtV;} |
| 98 | return RdV; |
| 99 | } |
| 100 | |
| 101 | Note that generate_A2_add updates the disassembly context to be processed |
| 102 | when the packet commits (see "Packet Semantics" below). |
| 103 | |
| 104 | The generator checks for fGEN_TCG_<tag> macro. This allows us to generate |
| 105 | TCG code instead of a call to the helper. If defined, the macro takes 1 |
| 106 | argument. |
| 107 | C semantics (aka short code) |
| 108 | |
| 109 | This allows the code generator to override the auto-generated code. In some |
| 110 | cases this is necessary for correct execution. We can also override for |
| 111 | faster emulation. For example, calling a helper for add is more expensive |
| 112 | than generating a TCG add operation. |
| 113 | |
| 114 | The gen_tcg.h file has any overrides. For example, we could write |
| 115 | #define fGEN_TCG_A2_add(GENHLPR, SHORTCODE) \ |
| 116 | tcg_gen_add_tl(RdV, RsV, RtV) |
| 117 | |
| 118 | The instruction semantics C code relies heavily on macros. In cases where the |
| 119 | C semantics are specified only with macros, we can override the default with |
| 120 | the short semantics option and #define the macros to generate TCG code. One |
| 121 | example is L2_loadw_locked: |
| 122 | Instruction tag L2_loadw_locked |
| 123 | Assembly syntax "Rd32=memw_locked(Rs32)" |
| 124 | Instruction semantics "{ fEA_REG(RsV); fLOAD_LOCKED(1,4,u,EA,RdV) }" |
| 125 | |
| 126 | In gen_tcg.h, we use the shortcode |
| 127 | #define fGEN_TCG_L2_loadw_locked(SHORTCODE) \ |
| 128 | SHORTCODE |
| 129 | |
| 130 | There are also cases where we brute force the TCG code generation. |
| 131 | Instructions with multiple definitions are examples. These require special |
| 132 | handling because qemu helpers can only return a single value. |
| 133 | |
| 134 | For HVX vectors, the generator behaves slightly differently. The wide vectors |
| 135 | won't fit in a TCGv or TCGv_i64, so we pass TCGv_ptr variables to pass the |
| 136 | address to helper functions. Here's an example for an HVX vector-add-word |
| 137 | istruction. |
| 138 | static void generate_V6_vaddw(DisasContext *ctx) |
| 139 | { |
| 140 | Insn *insn __attribute__((unused)) = ctx->insn; |
| 141 | const int VdN = insn->regno[0]; |
| 142 | const intptr_t VdV_off = |
| 143 | ctx_future_vreg_off(ctx, VdN, 1, true); |
| 144 | TCGv_ptr VdV = tcg_temp_new_ptr(); |
| 145 | tcg_gen_addi_ptr(VdV, tcg_env, VdV_off); |
| 146 | const int VuN = insn->regno[1]; |
| 147 | const intptr_t VuV_off = |
| 148 | vreg_src_off(ctx, VuN); |
| 149 | TCGv_ptr VuV = tcg_temp_new_ptr(); |
| 150 | const int VvN = insn->regno[2]; |
| 151 | const intptr_t VvV_off = |
| 152 | vreg_src_off(ctx, VvN); |
| 153 | TCGv_ptr VvV = tcg_temp_new_ptr(); |
| 154 | tcg_gen_addi_ptr(VuV, tcg_env, VuV_off); |
| 155 | tcg_gen_addi_ptr(VvV, tcg_env, VvV_off); |
| 156 | gen_helper_V6_vaddw(tcg_env, VdV, VuV, VvV); |
| 157 | } |
| 158 | |
| 159 | Notice that we also generate a variable named <operand>_off for each operand of |
| 160 | the instruction. This makes it easy to override the instruction semantics with |
| 161 | functions from tcg-op-gvec.h. Here's the override for this instruction. |
| 162 | #define fGEN_TCG_V6_vaddw(SHORTCODE) \ |
| 163 | tcg_gen_gvec_add(MO_32, VdV_off, VuV_off, VvV_off, \ |
| 164 | sizeof(MMVector), sizeof(MMVector)) |
| 165 | |
| 166 | Finally, we notice that the override doesn't use the TCGv_ptr variables, so |
| 167 | we don't generate them when an override is present. Here is what we generate |
| 168 | when the override is present. |
| 169 | static void generate_V6_vaddw(DisasContext *ctx) |
| 170 | { |
| 171 | Insn *insn __attribute__((unused)) = ctx->insn; |
| 172 | const int VdN = insn->regno[0]; |
| 173 | const intptr_t VdV_off = |
| 174 | ctx_future_vreg_off(ctx, VdN, 1, true); |
| 175 | const int VuN = insn->regno[1]; |
| 176 | const intptr_t VuV_off = |
| 177 | vreg_src_off(ctx, VuN); |
| 178 | const int VvN = insn->regno[2]; |
| 179 | const intptr_t VvV_off = |
| 180 | vreg_src_off(ctx, VvN); |
| 181 | fGEN_TCG_V6_vaddw({ fHIDE(int i;) fVFOREACH(32, i) { VdV.w[i] = VuV.w[i] + VvV.w[i] ; } }); |
| 182 | } |
| 183 | |
| 184 | We also generate an analyze_<tag> function for each instruction. Currently, |
| 185 | these functions record the reads and writes to registers by calling ctx_log_*. |
| 186 | During gen_start_packet, we invoke the analyze_<tag> function for each instruction in |
| 187 | the packet, and we mark the implicit writes. The analysis determines if the packet |
| 188 | semantics can be short-circuited. If not, we initialize the result register for each |
| 189 | of the predicated assignments. |
| 190 | |
| 191 | In addition to instruction semantics, we use a generator to create the decode |
| 192 | tree. This generation is a four step process. |
| 193 | Step 1 is to run target/hexagon/gen_dectree_import.c to produce |
| 194 | <BUILD_DIR>/target/hexagon/iset.py |
| 195 | Step 2 is to import iset.py into target/hexagon/gen_decodetree.py to produce |
| 196 | <BUILD_DIR>/target/hexagon/normal_decode_generated |
| 197 | <BUILD_DIR>/target/hexagon/hvx_decode_generated |
| 198 | <BUILD_DIR>/target/hexagon/subinsn_*_decode_generated |
| 199 | Step 3 is to process the above files with QEMU's decodetree.py to produce |
| 200 | <BUILD_DIR>/target/hexagon/decode_*_generated.c.inc |
| 201 | Step 4 is to import iset.py into target/hexagon/gen_trans_funcs.py to produce |
| 202 | <BUILD_DIR>/target/hexagon/decodetree_trans_funcs_generated.c.inc |
| 203 | |
| 204 | *** Key Files *** |
| 205 | |
| 206 | cpu.h |
| 207 | |
| 208 | This file contains the definition of the CPUHexagonState struct. It is the |
| 209 | runtime information for each thread and contains stuff like the GPR and |
| 210 | predicate registers. |
| 211 | |
| 212 | macros.h |
| 213 | mmvec/macros.h |
| 214 | |
| 215 | The Hexagon arch lib relies heavily on macros for the instruction semantics. |
| 216 | This is a great advantage for qemu because we can override them for different |
| 217 | purposes. You will also notice there are sometimes two definitions of a macro. |
| 218 | The QEMU_GENERATE variable determines whether we want the macro to generate TCG |
| 219 | code. If QEMU_GENERATE is not defined, we want the macro to generate vanilla |
| 220 | C code that will work in the helper implementation. |
| 221 | |
| 222 | translate.c |
| 223 | |
| 224 | The functions in this file generate TCG code for a translation block. Some |
| 225 | important functions in this file are |
| 226 | |
| 227 | gen_start_packet - initialize the data structures for packet semantics |
| 228 | gen_commit_packet - commit the register writes, stores, etc for a packet |
| 229 | decode_and_translate_packet - disassemble a packet and generate code |
| 230 | |
| 231 | genptr.c |
| 232 | gen_tcg.h |
| 233 | |
| 234 | These files create a function for each instruction. It is mostly composed of |
| 235 | fGEN_TCG_<tag> definitions followed by including tcg_funcs_generated.c.inc. |
| 236 | |
| 237 | op_helper.c |
| 238 | |
| 239 | This file contains the implementations of all the helpers. There are a few |
| 240 | general purpose helpers, but most of them are generated by including |
| 241 | helper_funcs_generated.c.inc. There are also several helpers used for debugging. |
| 242 | |
| 243 | |
| 244 | *** Packet Semantics *** |
| 245 | |
| 246 | VLIW packet semantics differ from serial semantics in that all input operands |
| 247 | are read, then the operations are performed, then all the results are written. |
| 248 | For example, this packet performs a swap of registers r0 and r1 |
| 249 | { r0 = r1; r1 = r0 } |
| 250 | Note that the result is different if the instructions are executed serially. |
| 251 | |
| 252 | Packet semantics dictate that we defer any changes of state until the entire |
| 253 | packet is committed. We record the results of each instruction in a side data |
| 254 | structure, and update the visible processor state when we commit the packet. |
| 255 | |
| 256 | The data structures are divided between the runtime state and the translation |
| 257 | context. |
| 258 | |
| 259 | During the TCG generation (see translate.[ch]), we use the DisasContext to |
| 260 | track what needs to be done during packet commit. Here are the relevant |
| 261 | fields |
| 262 | |
| 263 | reg_log list of registers written |
| 264 | reg_log_idx index into ctx_reg_log |
| 265 | pred_log list of predicates written |
| 266 | pred_log_idx index into ctx_pred_log |
| 267 | store_width width of stores (indexed by slot) |
| 268 | |
| 269 | During runtime, the following fields in CPUHexagonState (see cpu.h) are used |
| 270 | |
| 271 | new_value new value of a given register |
| 272 | reg_written boolean indicating if register was written |
| 273 | new_pred_value new value of a predicate register |
| 274 | pred_written boolean indicating if predicate was written |
| 275 | mem_log_stores record of the stores (indexed by slot) |
| 276 | |
| 277 | For Hexagon Vector eXtensions (HVX), the following fields are used |
| 278 | VRegs Vector registers |
| 279 | future_VRegs Registers to be stored during packet commit |
| 280 | tmp_VRegs Temporary registers *not* stored during commit |
| 281 | QRegs Q (vector predicate) registers |
| 282 | future_QRegs Registers to be stored during packet commit |
| 283 | |
| 284 | *** Debugging *** |
| 285 | |
| 286 | To track down nasty issues with Hexagon->TCG generation, we compare the |
| 287 | execution results with actual hardware running on a Hexagon Linux target. |
| 288 | Run qemu with the "-d cpu" option. Then, we can diff the results and figure |
| 289 | out where qemu and hardware behave differently. |
| 290 | |
| 291 | The stacks are located at different locations. We handle this by changing |
| 292 | env->stack_adjust in translate.c. First, set this to zero and run qemu. |
| 293 | Then, change env->stack_adjust to the difference between the two stack |
| 294 | locations. Then rebuild qemu and run again. That will produce a very |
| 295 | clean diff. |
| 296 | |
| 297 | Here are some handy places to set breakpoints |
| 298 | |
| 299 | At the call to gen_start_packet for a given PC (note that the line number |
| 300 | might change in the future) |
| 301 | br translate.c:602 if ctx->base.pc_next == 0xdeadbeef |
| 302 | The helper function for each instruction is named helper_<TAG>, so here's |
| 303 | an example that will set a breakpoint at the start |
| 304 | br helper_A2_add |