GCC Code Coverage Report

Directory: avs_core/
Coverage: low: ≥ 0% medium: ≥ 75.0% high: ≥ 90.0%
Coverage Exec / Excl / Total
Lines: 25.3% 1282 / 0 / 5070
Functions: 20.7% 57 / 0 / 276
Branches: 3.4% 1549 / 0 / 45776
filters/exprfilter/exprfilter.cpp
Line Branch Exec Source
1 /*
2 *
3 * Avisynth+ Expression filter, ported from the VapourSynth project
4 * Copyright (c) 2012-2015 Fredrik Mellbin
5 *
6 * Additions and differences to VS r39 version:
7 * ------------------------------
8 * (similar features to the masktools mt_lut family syntax)
9 * Operator aliases:
10 * Caret (^) can be used like pow
11 * For equality check "==" can be used like "="
12 * & same as and
13 * | same as or
14 * New operator: != (not equal)
15 * Built-in constants
16 * ymin, ymax (ymin_a .. ymin_z for individual clips) - the usual luma limits (16..235 or scaled equivalents)
17 * cmin, cmax (cmin_a .. cmin_z) - chroma limits (16..240 or scaled equivalents)
18 * range_half (range_half_a .. range_half_z) - half of the range, (128 or scaled equivalents)
19 * range_size, range_half, range_min, range_max (range_size_a .. range_size_z , etc..)
20 * Autoscale helper functions (operand is treated as being a 8 bit constant unless i8..i16 or f32 is specified)
21 * scaleb (scale by bit shift - mul or div by 2, 4, 6, 8...)
22 * scalef (scale by stretch full scale - mul or div by source_max/target_max
23 * Keywords for modifying base bit depth for scaleb and scalef
24 * i8, i10, i12, i14, i16, f32
25 * Built-in math constant
26 * pi
27 * Alpha plane handling. When no separate expression is supplied for alpha, plane is copied instead of reusing last expression parameter
28 * Proper clamping when storing 10,12 or 14 bit outputs
29 * Faster storing of results for 8 and 10-16 bit outputs
30 * 16 pixels/cycle instead of 8 when avx2, with fallback to 8-pixel case on the right edge. Thus no need for 64 byte alignment for 32 bit float.
31 * (Load zeros for nonvisible pixels, when simd block size goes beyond image width, to prevent garbage input for simd calculation)
32 * Optimizations: x^0.5 is sqrt, ^1 +0 -0 *1 /1 to nothing, ^2, ^3, ^4 is done by faster and more precise multiplication
33 * spatial input variables in expr syntax:
34 * sx, sy (absolute x and y coordinates, 0 to width-1 and 0 to height-1)
35 * sxr, syr (relative x and y coordinates, from 0 to 1.0)
36 * Optimize: recognize constant plane expression: use fast memset instead of generic simd process. Approx. 3-4x (32 bits) to 10-12x (8 bits) speedup
37 * Optimize: Recognize single clip letter in expression: use fast plane copy (BitBlt)
38 * (e.g. for 8-16 bits: instead of load-convert_to_float-clamp-convert_to_int-store). Approx. 1.4x (32 bits), 3x (16 bits), 8-9x (8 bits) speedup
39 * Optimize: do not call GetFrame for input clips that are not referenced or plane-copied
40 * 20171211: Implement relative pixel indexing e.g. x[-1,-3], requires SSSE3
41 * Fix jitasm code generation (corrupt code when doing register reorders)
42 * 20171212: Variables: A..Z
43 * To store the current top value of the stack to a variable: A@ .. Z@
44 * To store the current top value and pop it from the top of the stack: A^.. Z^
45 * To use a stored variable: single uppercase letter. E.g. A
46 * 20171214: Trig.functions (C only): sin, cos, tan, asin, acos, atan
47 * '%' The implementation is fmod-like: x - trunc(x/d)*d.
48 * Note: SSE2 and up is using trunc for float->integer conversion, works for usual width/height magnitude.
49 * (A float can hold a 24 bit integer w/o losing precision)
50 * expr constants: 'width', 'height' for current plane width and height
51 * expr auto variable: 'frameno' holds the current frame number 0..total number of frames-1
52 * expr auto variable: 'time' relative time in clip, 0 <= time <= 1
53 * calculation: time = frameno/(total number of frames - 1)
54 * 20180614 new parameters: scale_inputs, clamp_float
55 * implement 'clip' three operand operator like in masktools2: x minvalue maxvalue clip -> max(min(x, maxvalue), minvalue)
56 * 20191120 yrange_min, yrange_half, yrange_max, clamp_float_UV param, allow "float_UV" for scale_inputs
57 * yscalef, yscaleb forcing non-chroma rules for scaling even when processing chroma planes
58 * 202107xx round, floor, ceil, trunc (acceleration from SSE4.1 and up)
59 * arbitrary variable names; instead of 'A' to 'Z', up to 256 of them can be used
60 * 20210924 frame property access clip.framePropName syntax after VSAkarin idea (no array access yet)
61 * sin and cos as SIMD (VSAkarin, port from VS)
62 * 20211116 neg
63 * 20211117 atan2 as SIMD
64 * 20211118 sgn
65 * 20211127 lutx, lutxy (lut=1 and 2)
66 * 20211128 allow f32 for 'scale_inputs' when "int", "intf", "all", "allf"
67 * 20250309 tan as SIMD
68 *
69 * Differences from masktools 2.2.15
70 * ---------------------------------
71 * Up to 26 clips are allowed (x,y,z,a,b,...w). Masktools handles only up to 4 clips with its mt_lut, my_lutxy, mt_lutxyz, mt_lutxyza
72 * Clips with different bit depths are allowed
73 * works with 32 bit floats instead of 64 bit double internally
74 * less functions (e.g. no bit shifts)
75 * logical 'false' is 0 instead of -1
76 * avs+: ymin, ymax, etc built-in constants can have a _X suffix, where X is the corresponding clip designator letter. E.g. cmax_z, range_half_x
77 * mt_lutspa-like functionality is available through "sx", "sy", "sxr", "syr"
78 */
79
80 #include <iostream>
81 #include <locale>
82 #include <sstream>
83 #include <vector>
84 #include <list>
85 #include <string>
86 #include <algorithm>
87 #include <stdexcept>
88 #include <memory>
89 #include <cmath>
90 #include <unordered_map>
91
92 #include <avisynth.h>
93
94 #ifdef AVS_WINDOWS
95 #include <avs/win.h>
96 #else
97 #include <avs/posix.h>
98 #endif
99
100 #include <stdlib.h>
101 #include "../../core/internal.h"
102 #include "../../convert/convert_planar.h" // fill_plane
103 #include "../../convert/convert_helper.h"
104 #include "avs/alignment.h"
105
106 #if defined(_MSC_VER) || (defined(__clang__) && defined(_MSC_VER)) || defined(__MINGW32__) || defined(__MINGW64__)
107 #include <malloc.h> // For _aligned_malloc and _aligned_free
108 #endif
109
110 #if (defined(_WIN64) && (defined(_M_AMD64) || defined(_M_X64))) || defined(__x86_64__)
111 #define JITASM64
112 #endif
113
114 #ifdef INTEL_INTRINSICS
115 #define VS_TARGET_CPU_X86
116 #endif
117 #ifdef AVS_WINDOWS
118 #define VS_TARGET_OS_WINDOWS
119 #endif
120 #include "exprfilter.h"
121
122 #ifdef VS_TARGET_CPU_X86
123 #ifndef NOMINMAX
124 #define NOMINMAX
125 #endif
126 #include "jitasm.h"
127 #endif
128
129 #ifndef VS_TARGET_OS_WINDOWS
130 #include <sys/mman.h>
131 #endif
132
133 #ifdef XP_TLS
134 #ifdef MSVC_PURE
135 // v141_xp workaround: disabling /O2 global optimizations reduces build time
136 // from 56 minutes to 48 seconds.
137 #pragma optimize("g", off) // disables global optimizations (e.g. /O2 max opt,speed)
138 #pragma optimize("t", on) // favor speed /Ot
139 #pragma auto_inline(on) // enable aggressive inlining, equivalent of /Ob2
140 #endif
141 #endif
142
143 #ifdef VS_TARGET_CPU_X86
144
145 //#define TEST_AVX2_CODEGEN_IN_AVX
146
147 #include <immintrin.h>
148
149 #if defined(GCC) || defined(CLANG)
150 #include <avxintrin.h>
151 #endif
152
153 // rounder constants
154 constexpr int FROUND_TO_NEAREST_INT = 0x00;
155 constexpr int FROUND_TO_NEG_INF = 0x01;
156 constexpr int FROUND_TO_POS_INF = 0x02;
157 constexpr int FROUND_TO_ZERO = 0x03;
158 constexpr int FROUND_NO_EXC = 0x08;
159
160 // normal versions work with two xmm or ymm registers (2*4 or 2*8 pixels per cycle)
161 // _Single suffixed versions work only one xmm or ymm registers at a time (1*4 or 1*8 pixels per cycle)
162
163 #define OneArgOp(instr) \
164 auto &t1 = stack.back(); \
165 instr(t1.first, t1.first); \
166 instr(t1.second, t1.second);
167
168 #define OneArgOp_Single(instr) \
169 auto &t1 = stack1.back(); \
170 instr(t1, t1);
171
172 #define TwoArgOp(instr) \
173 auto t1 = stack.back(); \
174 stack.pop_back(); \
175 auto &t2 = stack.back(); \
176 instr(t2.first, t1.first); \
177 instr(t2.second, t1.second);
178
179 #define TwoArgOp_Single(instr) \
180 auto t1 = stack1.back(); \
181 stack1.pop_back(); \
182 auto &t2 = stack1.back(); \
183 instr(t2, t1);
184
185 #define TwoArgOp_Avx(instr) \
186 auto t1 = stack.back(); \
187 stack.pop_back(); \
188 auto &t2 = stack.back(); \
189 instr(t2.first, t2.first, t1.first); \
190 instr(t2.second, t2.second, t1.second);
191
192 #define TwoArgOp_Single_Avx(instr) \
193 auto t1 = stack1.back(); \
194 stack1.pop_back(); \
195 auto &t2 = stack1.back(); \
196 instr(t2, t2, t1);
197
198 #define CmpOp(instr) \
199 auto t1 = stack.back(); \
200 stack.pop_back(); \
201 auto t2 = stack.back(); \
202 stack.pop_back(); \
203 instr(t1.first, t2.first); \
204 instr(t1.second, t2.second); \
205 andps(t1.first, CPTR(elfloat_one)); \
206 andps(t1.second, CPTR(elfloat_one)); \
207 stack.push_back(t1);
208
209 #define CmpOp_Single(instr) \
210 auto t1 = stack1.back(); \
211 stack1.pop_back(); \
212 auto t2 = stack1.back(); \
213 stack1.pop_back(); \
214 instr(t1, t2); \
215 andps(t1, CPTR(elfloat_one)); \
216 stack1.push_back(t1);
217
218 #define CmpOp_Avx(instr, op) \
219 auto t1 = stack.back(); \
220 stack.pop_back(); \
221 auto t2 = stack.back(); \
222 stack.pop_back(); \
223 instr(t1.first, t1.first, t2.first, op); \
224 instr(t1.second, t1.second, t2.second, op); \
225 vandps(t1.first, t1.first, CPTR_AVX(elfloat_one)); \
226 vandps(t1.second, t1.second, CPTR_AVX(elfloat_one)); \
227 stack.push_back(t1);
228
229 #define CmpOp_Single_Avx(instr, op) \
230 auto t1 = stack1.back(); \
231 stack1.pop_back(); \
232 auto t2 = stack1.back(); \
233 stack1.pop_back(); \
234 instr(t1, t1, t2, op); \
235 vandps(t1, t1, CPTR_AVX(elfloat_one)); \
236 stack1.push_back(t1);
237
238 #define LogicOp(instr) \
239 auto t1 = stack.back(); \
240 stack.pop_back(); \
241 auto t2 = stack.back(); \
242 stack.pop_back(); \
243 cmpnleps(t1.first, zero); \
244 cmpnleps(t1.second, zero); \
245 cmpnleps(t2.first, zero); \
246 cmpnleps(t2.second, zero); \
247 instr(t1.first, t2.first); \
248 instr(t1.second, t2.second); \
249 andps(t1.first, CPTR(elfloat_one)); \
250 andps(t1.second, CPTR(elfloat_one)); \
251 stack.push_back(t1);
252
253 #define LogicOp_Single(instr) \
254 auto t1 = stack1.back(); \
255 stack1.pop_back(); \
256 auto t2 = stack1.back(); \
257 stack1.pop_back(); \
258 cmpnleps(t1, zero); \
259 cmpnleps(t2, zero); \
260 instr(t1, t2); \
261 andps(t1, CPTR(elfloat_one)); \
262 stack1.push_back(t1);
263
264 #define LogicOp_Avx(instr) \
265 auto t1 = stack.back(); \
266 stack.pop_back(); \
267 auto t2 = stack.back(); \
268 stack.pop_back(); \
269 vcmpps(t1.first, t1.first, zero, _CMP_GT_OQ); \
270 vcmpps(t1.second, t1.second, zero, _CMP_GT_OQ); \
271 vcmpps(t2.first, t2.first, zero, _CMP_GT_OQ); \
272 vcmpps(t2.second, t2.second, zero, _CMP_GT_OQ); \
273 instr(t1.first, t1.first, t2.first); \
274 instr(t1.second, t1.second, t2.second); \
275 vandps(t1.first, t1.first, CPTR_AVX(elfloat_one)); \
276 vandps(t1.second, t1.second, CPTR_AVX(elfloat_one)); \
277 stack.push_back(t1);
278
279 #define LogicOp_Single_Avx(instr) \
280 auto t1 = stack1.back(); \
281 stack1.pop_back(); \
282 auto t2 = stack1.back(); \
283 stack1.pop_back(); \
284 vcmpps(t1, t1, zero, _CMP_GT_OQ); \
285 vcmpps(t2, t2, zero, _CMP_GT_OQ); \
286 instr(t1, t1, t2); \
287 vandps(t1, t1, CPTR_AVX(elfloat_one)); \
288 stack1.push_back(t1);
289
290 enum {
291 elabsmask, elc7F, elmin_norm_pos, elinv_mant_mask,
292 elfloat_one, elfloat_minusone, elfloat_half, elsignmask, elstore8, elstore10, elstore12, elstore14, elstore16,
293 spatialX, spatialX2,
294 loadmask1000, loadmask1100, loadmask1110,
295 elShuffleForRight0, elShuffleForRight1, elShuffleForRight2, elShuffleForRight3, elShuffleForRight4, elShuffleForRight5, elShuffleForRight6,
296 elShuffleForLeft0, elShuffleForLeft1, elShuffleForLeft2, elShuffleForLeft3, elShuffleForLeft4, elShuffleForLeft5, elShuffleForLeft6,
297 elexp_hi, elexp_lo, elcephes_LOG2EF,
298 elcephes_exp_C1, elcephes_log_q2 = elcephes_exp_C1, elcephes_exp_C2, elcephes_log_q1 = elcephes_exp_C2, elcephes_exp_p0, elcephes_exp_p1, elcephes_exp_p2, elcephes_exp_p3, elcephes_exp_p4, elcephes_exp_p5, elcephes_SQRTHF,
299 elcephes_log_p0, elcephes_log_p1, elcephes_log_p2, elcephes_log_p3, elcephes_log_p4, elcephes_log_p5, elcephes_log_p6, elcephes_log_p7, elcephes_log_p8,
300 float_invpi, float_rintf,
301 float_pi1, float_pi2, float_pi3, float_pi4,
302 float_sinC3, float_sinC5, float_sinC7, float_sinC9,
303 float_cosC2, float_cosC4, float_cosC6, float_cosC8,
304 float_atan2f_rmul, float_atan2f_radd, float_atan2f_tmul, float_atan2f_tadd, float_atan2f_halfpi, float_atan2f_pi,
305 float_tan_p0, float_tan_p2, float_tan_p4, float_tan_p6, float_tan_p8,
306 float_tan_q0, float_tan_q2, float_tan_q4, float_tan_q6, float_tan_q8,
307 float_tan_small_limit, float_tan_asympt_a1, float_tan_asympt_a2, float_tan_asympt_limit
308 };
309
310 // constants for xmm
311
312 #define XCONST(x) { x, x, x, x }
313 #define MAKEDWORD(ch0, ch1, ch2, ch3) \
314 ((uint32_t)(unsigned char)(ch0) | ((uint32_t)(unsigned char)(ch1) << 8) | \
315 ((uint32_t)(unsigned char)(ch2) << 16) | ((uint32_t)(unsigned char)(ch3) << 24 ))
316 #define XBYTECONST(a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15) \
317 { (int)MAKEDWORD(a0,a1,a2,a3), (int)MAKEDWORD(a4,a5,a6,a7), (int)MAKEDWORD(a8,a9,a10,a11), (int)MAKEDWORD(a12,a13,a14,a15) }
318
319
320 static constexpr ExprUnion logexpconst alignas(16)[87][4] = {
321 XCONST(0x7FFFFFFF), // absmask
322 XCONST(0x7F), // c7F
323 XCONST(0x00800000), // min_norm_pos
324 XCONST(~0x7f800000), // inv_mant_mask
325 XCONST(1.0f), // float_one
326 XCONST(-1.0f), // float_minusone
327 XCONST(0.5f), // float_half
328 XCONST(0x80000000), // elsignmask
329 XCONST(255.0f), // store8
330 XCONST(1023.0f), // store10 (avs+)
331 XCONST(4095.0f), // store12 (avs+)
332 XCONST(16383.0f), // store14 (avs+)
333 XCONST(65535.0f), // store16
334 { 0.0f, 1.0f, 2.0f, 3.0f }, // spatialX
335 { 4.0f, 5.0f, 6.0f, 7.0f }, // spatialX2
336 { (int)0xFFFFFFFF, 0x00000000, 0x00000000, 0x00000000 }, // loadmask1000
337 { (int)0xFFFFFFFF, (int)0xFFFFFFFF, 0x00000000, 0x00000000 }, // loadmask1100
338 { (int)0xFFFFFFFF, (int)0xFFFFFFFF, (int)0xFFFFFFFF, 0x00000000 }, // loadmask1110
339 XBYTECONST(0,1,2,3,4,5,6,7, 8,9,10,11,12,13,12,13), // elShuffleForRight0
340 XBYTECONST(0,1,2,3,4,5,6,7, 8,9,10,11,10,11,10,11), // elShuffleForRight1
341 XBYTECONST(0,1,2,3,4,5,6,7, 8,9,8,9,8,9,8,9), // elShuffleForRight2
342 XBYTECONST(0,1,2,3,4,5,6,7, 6,7,6,7,6,7,6,7), // elShuffleForRight3
343 XBYTECONST(0,1,2,3,4,5,4,5, 4,5,4,5,4,5,4,5), // elShuffleForRight4
344 XBYTECONST(0,1,2,3,2,3,2,3, 2,3,2,3,2,3,2,3), // elShuffleForRight5
345 XBYTECONST(0,1,0,1,0,1,0,1, 0,1,0,1,0,1,0,1), // elShuffleForRight6
346 XBYTECONST(2,3,2,3,4,5,6,7, 8,9,10,11,12,13,14,15), // elShuffleForLeft0
347 XBYTECONST(4,5,4,5,4,5,6,7, 8,9,10,11,12,13,14,15), // elShuffleForLeft1
348 XBYTECONST(6,7,6,7,6,7,6,7, 8,9,10,11,12,13,14,15), // elShuffleForLeft2
349 XBYTECONST(8,9,8,9,8,9,8,9, 8,9,10,11,12,13,14,15), // elShuffleForLeft3
350 XBYTECONST(10,11,10,11,10,11,10,11, 10,11,10,11,12,13,14,15), // elShuffleForLeft4
351 XBYTECONST(12,13,12,13,12,13,12,13, 12,13,12,13,12,13,14,15), // elShuffleForLeft5
352 XBYTECONST(14,15,14,15,14,15,14,15, 14,15,14,15,14,15,14,15), // elShuffleForLeft6
353 XCONST(88.3762626647949f), // exp_hi
354 XCONST(-88.3762626647949f), // exp_lo
355 XCONST(1.44269504088896341f), // cephes_LOG2EF
356 XCONST(0.693359375f), // cephes_exp_C1
357 XCONST(-2.12194440e-4f), // cephes_exp_C2
358 XCONST(1.9875691500E-4f), // cephes_exp_p0
359 XCONST(1.3981999507E-3f), // cephes_exp_p1
360 XCONST(8.3334519073E-3f), // cephes_exp_p2
361 XCONST(4.1665795894E-2f), // cephes_exp_p3
362 XCONST(1.6666665459E-1f), // cephes_exp_p4
363 XCONST(5.0000001201E-1f), // cephes_exp_p5
364 XCONST(0.707106781186547524f), // cephes_SQRTHF
365 XCONST(7.0376836292E-2f), // cephes_log_p0
366 XCONST(-1.1514610310E-1f), // cephes_log_p1
367 XCONST(1.1676998740E-1f), // cephes_log_p2
368 XCONST(-1.2420140846E-1f), // cephes_log_p3
369 XCONST(+1.4249322787E-1f), // cephes_log_p4
370 XCONST(-1.6668057665E-1f), // cephes_log_p5
371 XCONST(+2.0000714765E-1f), // cephes_log_p6
372 XCONST(-2.4999993993E-1f), // cephes_log_p7
373 XCONST(+3.3333331174E-1f), // cephes_log_p8
374 XCONST(0x3ea2f983), // float_invpi, 1/pi = 0.31830988618379067154f
375 XCONST(0x4b400000), // float_rintf Used for rounding
376 XCONST(0x40490000), // float_pi1 High precision part of Pi
377 XCONST(0x3a7da000), // float_pi2 Second part of Pi for extended precision
378 XCONST(0x34222000), // float_pi3 Third part of Pi for extended precision
379 XCONST(0x2cb4611a), // float_pi4 Fourth part of Pi for extended precision
380 XCONST(0xbe2aaaa6), // float_sinC3
381 XCONST(0x3c08876a), // float_sinC5
382 XCONST(0xb94fb7ff), // float_sinC7
383 XCONST(0x362edef8), // float_sinC9
384 XCONST(static_cast<int32_t>(0xBEFFFFE2)), // float_cosC2
385 XCONST(0x3D2AA73C), // float_cosC4
386 XCONST(static_cast<int32_t>(0XBAB58D50)), // float_cosC6
387 XCONST(0x37C1AD76), // float_cosC8
388 XCONST(0x3ccb7dda), // float_atan2f_rmul 0.024840285f
389 XCONST(0x3e3f4c37), // float_atan2f_radd 0.18681418f
390 XCONST(0xbdc0b66d), // float_atan2f_tmul -0.094097948f
391 XCONST(0xbeaa0d0a), // float_atan2f_tadd -0.33213072f
392 XCONST(0x3fc90fdb), // float_atan2f_halfpi 1.57079637f
393 XCONST(0x40490fdb), // float_atan2f_pi 3.14159274f
394 // Tangent approximation coefficients up to p8 and helpers
395 XCONST(1.0f), // float_tan_p0 - Numerator constant term
396 XCONST(0.3333314036f), // float_tan_p2 - Numerator x^2 coefficient
397 XCONST(0.1333923995f), // float_tan_p4 - Numerator x^4 coefficient
398 XCONST(0.0533740603f), // float_tan_p6 - Numerator x^6 coefficient
399 XCONST(0.0245650893f), // float_tan_p8 - Numerator x^8 coefficient
400 XCONST(1.0f), // float_tan_q0 - Denominator constant term
401 XCONST(0.1333835001f), // float_tan_q2 - Denominator x^2 coefficient
402 XCONST(0.0089270802f), // float_tan_q4 - Denominator x^4 coefficient
403 XCONST(0.0005908960f), // float_tan_q6 - Denominator x^6 coefficient
404 XCONST(0.0000342237f), // float_tan_q8 - Denominator x^8 coefficient
405 XCONST(1e-4f), // float_tan_small_limit
406 XCONST(0.97f), // float_tan_asympt_a1
407 XCONST(0.35f), // float_tan_asympt_a2
408 XCONST(0.8f), // float_tan_asympt_limit
409 };
410
411
412 #define CPTR(x) (xmmword_ptr[constptr + (x) * 16])
413
414 // AVX2 stuff
415 // constants for ymm
416
417 #undef XCONST
418 #define XCONST(x) { x, x, x, x, x, x, x, x }
419
420 static constexpr ExprUnion logexpconst_avx alignas(32)[87][8] = {
421 XCONST(0x7FFFFFFF), // absmask
422 XCONST(0x7F), // c7F
423 XCONST(0x00800000), // min_norm_pos
424 XCONST(~0x7f800000), // inv_mant_mask
425 XCONST(1.0f), // float_one
426 XCONST(-1.0f), // float_minusone
427 XCONST(0.5f), // float_half
428 XCONST(0x80000000), // elsignmask
429 XCONST(255.0f), // store8
430 XCONST(1023.0f), // store10 (avs+)
431 XCONST(4095.0f), // store12 (avs+)
432 XCONST(16383.0f), // store14 (avs+)
433 XCONST(65535.0f), // store16
434 { 0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f }, // spatialX
435 { 8.0f, 9.0f, 10.0f, 11.0f, 12.0f, 13.0f, 14.0f, 15.0f }, // spatialX2
436 { (int)0xFFFFFFFF, 0x00000000, 0x00000000, 0x00000000, 0, 0, 0, 0 }, // loadmask1000 not used, avx supports blendps
437 { (int)0xFFFFFFFF, (int)0xFFFFFFFF, 0x00000000, 0x00000000, 0, 0, 0, 0 }, // loadmask1100 not used, avx supports blendps
438 { (int)0xFFFFFFFF, (int)0xFFFFFFFF, (int)0xFFFFFFFF, 0x00000000, 0, 0, 0, 0 }, // loadmask1110 not used, avx supports blendps
439 XCONST(0), // n/a elShuffleForRight0
440 XCONST(0), // n/a elShuffleForRight1
441 XCONST(0), // n/a elShuffleForRight2
442 XCONST(0), // n/a elShuffleForRight3
443 XCONST(0), // n/a elShuffleForRight4
444 XCONST(0), // n/a elShuffleForRight5
445 XCONST(0), // n/a elShuffleForRight6
446 XCONST(0), // n/a elShuffleForLeft0
447 XCONST(0), // n/a elShuffleForLeft1
448 XCONST(0), // n/a elShuffleForLeft2
449 XCONST(0), // n/a elShuffleForLeft3
450 XCONST(0), // n/a elShuffleForLeft4
451 XCONST(0), // n/a elShuffleForLeft5
452 XCONST(0), // n/a elShuffleForLeft6
453 XCONST(88.3762626647949f), // exp_hi
454 XCONST(-88.3762626647949f), // exp_lo
455 XCONST(1.44269504088896341f), // cephes_LOG2EF
456 XCONST(0.693359375f), // cephes_exp_C1
457 XCONST(-2.12194440e-4f), // cephes_exp_C2
458 XCONST(1.9875691500E-4f), // cephes_exp_p0
459 XCONST(1.3981999507E-3f), // cephes_exp_p1
460 XCONST(8.3334519073E-3f), // cephes_exp_p2
461 XCONST(4.1665795894E-2f), // cephes_exp_p3
462 XCONST(1.6666665459E-1f), // cephes_exp_p4
463 XCONST(5.0000001201E-1f), // cephes_exp_p5
464 XCONST(0.707106781186547524f), // cephes_SQRTHF
465 XCONST(7.0376836292E-2f), // cephes_log_p0
466 XCONST(-1.1514610310E-1f), // cephes_log_p1
467 XCONST(1.1676998740E-1f), // cephes_log_p2
468 XCONST(-1.2420140846E-1f), // cephes_log_p3
469 XCONST(+1.4249322787E-1f), // cephes_log_p4
470 XCONST(-1.6668057665E-1f), // cephes_log_p5
471 XCONST(+2.0000714765E-1f), // cephes_log_p6
472 XCONST(-2.4999993993E-1f), // cephes_log_p7
473 XCONST(+3.3333331174E-1f), // cephes_log_p8
474 XCONST(0x3ea2f983), // float_invpi, 1/pi = 0.31830988618379067154f
475 XCONST(0x4b400000), // float_rintf Used for rounding
476 XCONST(0x40490000), // float_pi1 High precision part of Pi
477 XCONST(0x3a7da000), // float_pi2 Second part of Pi for extended precision
478 XCONST(0x34222000), // float_pi3 Third part of Pi for extended precision
479 XCONST(0x2cb4611a), // float_pi4 Fourth part of Pi for extended precision
480 XCONST(0xbe2aaaa6), // float_sinC3
481 XCONST(0x3c08876a), // float_sinC5
482 XCONST(0xb94fb7ff), // float_sinC7
483 XCONST(0x362edef8), // float_sinC9
484 XCONST(static_cast<int32_t>(0xBEFFFFE2)), // float_cosC2
485 XCONST(0x3D2AA73C), // float_cosC4
486 XCONST(static_cast<int32_t>(0XBAB58D50)), // float_cosC6
487 XCONST(0x37C1AD76), // float_cosC8
488 XCONST(0x3ccb7dda), // float_atan2f_rmul 0.024840285f
489 XCONST(0x3e3f4c37), // float_atan2f_radd 0.18681418f
490 XCONST(0xbdc0b66d), // float_atan2f_tmul -0.094097948f
491 XCONST(0xbeaa0d0a), // float_atan2f_tadd -0.33213072f
492 XCONST(0x3fc90fdb), // float_atan2f_halfpi 1.57079637f
493 XCONST(0x40490fdb), // float_atan2f_pi 3.14159274f
494 // Tangent approximation coefficients up to p8 and helpers
495 XCONST(1.0f), // float_tan_p0 - Numerator constant term
496 XCONST(0.3333314036f), // float_tan_p2 - Numerator x^2 coefficient
497 XCONST(0.1333923995f), // float_tan_p4 - Numerator x^4 coefficient
498 XCONST(0.0533740603f), // float_tan_p6 - Numerator x^6 coefficient
499 XCONST(0.0245650893f), // float_tan_p8 - Numerator x^8 coefficient
500 XCONST(1.0f), // float_tan_q0 - Denominator constant term
501 XCONST(0.1333835001f), // float_tan_q2 - Denominator x^2 coefficient
502 XCONST(0.0089270802f), // float_tan_q4 - Denominator x^4 coefficient
503 XCONST(0.0005908960f), // float_tan_q6 - Denominator x^6 coefficient
504 XCONST(0.0000342237f), // float_tan_q8 - Denominator x^8 coefficient
505 XCONST(1e-4f), // float_tan_small_limit
506 XCONST(0.97f), // float_tan_asympt_a1
507 XCONST(0.35f), // float_tan_asympt_a2
508 XCONST(0.8f), // float_tan_asympt_limit
509 };
510 #undef XCONST
511
512 #define CPTR_AVX(x) (ymmword_ptr[constptr + (x) * 32])
513
514 #define EXP_PS(x) { \
515 XmmReg fx, emm0, etmp, y, mask, z; \
516 minps(x, CPTR(elexp_hi)); \
517 maxps(x, CPTR(elexp_lo)); \
518 movaps(fx, x); \
519 mulps(fx, CPTR(elcephes_LOG2EF)); \
520 addps(fx, CPTR(elfloat_half)); \
521 cvttps2dq(emm0, fx); \
522 cvtdq2ps(etmp, emm0); \
523 movaps(mask, etmp); \
524 cmpnleps(mask, fx); \
525 andps(mask, CPTR(elfloat_one)); \
526 movaps(fx, etmp); \
527 subps(fx, mask); \
528 movaps(etmp, fx); \
529 mulps(etmp, CPTR(elcephes_exp_C1)); \
530 movaps(z, fx); \
531 mulps(z, CPTR(elcephes_exp_C2)); \
532 subps(x, etmp); \
533 subps(x, z); \
534 movaps(z, x); \
535 mulps(z, z); \
536 movaps(y, CPTR(elcephes_exp_p0)); \
537 mulps(y, x); \
538 addps(y, CPTR(elcephes_exp_p1)); \
539 mulps(y, x); \
540 addps(y, CPTR(elcephes_exp_p2)); \
541 mulps(y, x); \
542 addps(y, CPTR(elcephes_exp_p3)); \
543 mulps(y, x); \
544 addps(y, CPTR(elcephes_exp_p4)); \
545 mulps(y, x); \
546 addps(y, CPTR(elcephes_exp_p5)); \
547 mulps(y, z); \
548 addps(y, x); \
549 addps(y, CPTR(elfloat_one)); \
550 cvttps2dq(emm0, fx); \
551 paddd(emm0, CPTR(elc7F)); \
552 pslld(emm0, 23); \
553 mulps(y, emm0); \
554 x = y; }
555
556 #define LOG_PS(x) { \
557 XmmReg emm0, invalid_mask, mask, y, etmp, z; \
558 xorps(invalid_mask, invalid_mask); \
559 cmpnleps(invalid_mask, x); \
560 maxps(x, CPTR(elmin_norm_pos)); \
561 movaps(emm0, x); \
562 psrld(emm0, 23); \
563 andps(x, CPTR(elinv_mant_mask)); \
564 orps(x, CPTR(elfloat_half)); \
565 psubd(emm0, CPTR(elc7F)); \
566 cvtdq2ps(emm0, emm0); \
567 addps(emm0, CPTR(elfloat_one)); \
568 movaps(mask, x); \
569 cmpltps(mask, CPTR(elcephes_SQRTHF)); \
570 movaps(etmp, x); \
571 andps(etmp, mask); \
572 subps(x, CPTR(elfloat_one)); \
573 andps(mask, CPTR(elfloat_one)); \
574 subps(emm0, mask); \
575 addps(x, etmp); \
576 movaps(z, x); \
577 mulps(z, z); \
578 movaps(y, CPTR(elcephes_log_p0)); \
579 mulps(y, x); \
580 addps(y, CPTR(elcephes_log_p1)); \
581 mulps(y, x); \
582 addps(y, CPTR(elcephes_log_p2)); \
583 mulps(y, x); \
584 addps(y, CPTR(elcephes_log_p3)); \
585 mulps(y, x); \
586 addps(y, CPTR(elcephes_log_p4)); \
587 mulps(y, x); \
588 addps(y, CPTR(elcephes_log_p5)); \
589 mulps(y, x); \
590 addps(y, CPTR(elcephes_log_p6)); \
591 mulps(y, x); \
592 addps(y, CPTR(elcephes_log_p7)); \
593 mulps(y, x); \
594 addps(y, CPTR(elcephes_log_p8)); \
595 mulps(y, x); \
596 mulps(y, z); \
597 movaps(etmp, emm0); \
598 mulps(etmp, CPTR(elcephes_log_q1)); \
599 addps(y, etmp); \
600 mulps(z, CPTR(elfloat_half)); \
601 subps(y, z); \
602 mulps(emm0, CPTR(elcephes_log_q2)); \
603 addps(x, y); \
604 addps(x, emm0); \
605 orps(x, invalid_mask); }
606
607 #define EXP_PS_AVX(x) { \
608 YmmReg fx, emm0, etmp, y, mask, z; \
609 vminps(x, x, CPTR_AVX(elexp_hi)); \
610 vmaxps(x, x, CPTR_AVX(elexp_lo)); \
611 vmulps(fx, x, CPTR_AVX(elcephes_LOG2EF)); \
612 vaddps(fx, fx, CPTR_AVX(elfloat_half)); \
613 vcvttps2dq(emm0, fx); \
614 vcvtdq2ps(etmp, emm0); \
615 vcmpps(mask, etmp, fx, _CMP_GT_OQ); /* cmpnleps */ \
616 vandps(mask, mask, CPTR_AVX(elfloat_one)); \
617 vsubps(fx, etmp, mask); \
618 vfnmadd231ps(x, fx, CPTR_AVX(elcephes_exp_C1)); \
619 vfnmadd231ps(x, fx, CPTR_AVX(elcephes_exp_C2)); \
620 vmulps(z, x, x); \
621 vmovaps(y, CPTR_AVX(elcephes_exp_p0)); \
622 vfmadd213ps(y, x, CPTR_AVX(elcephes_exp_p1)); \
623 vfmadd213ps(y, x, CPTR_AVX(elcephes_exp_p2)); \
624 vfmadd213ps(y, x, CPTR_AVX(elcephes_exp_p3)); \
625 vfmadd213ps(y, x, CPTR_AVX(elcephes_exp_p4)); \
626 vfmadd213ps(y, x, CPTR_AVX(elcephes_exp_p5)); \
627 vfmadd213ps(y, z, x); \
628 vaddps(y, y, CPTR_AVX(elfloat_one)); \
629 vcvttps2dq(emm0, fx); \
630 vpaddd(emm0, emm0, CPTR_AVX(elc7F)); \
631 vpslld(emm0, emm0, 23); \
632 vmulps(x, y, emm0); \
633 }
634
635 #define LOG_PS_AVX(x) { \
636 YmmReg emm0, invalid_mask, mask, y, etmp, z; \
637 vcmpps(invalid_mask, zero, x, _CMP_GT_OQ); /* cmpnleps. or signalling _CMP_NLE_US? */ \
638 vmaxps(x, x, CPTR_AVX(elmin_norm_pos)); \
639 vpsrld(emm0, x, 23); \
640 vandps(x, x, CPTR_AVX(elinv_mant_mask)); \
641 vorps(x, x, CPTR_AVX(elfloat_half)); \
642 vpsubd(emm0, emm0, CPTR_AVX(elc7F)); \
643 vcvtdq2ps(emm0, emm0); \
644 vaddps(emm0, emm0, CPTR_AVX(elfloat_one)); \
645 vcmpps(mask, x, CPTR_AVX(elcephes_SQRTHF), _CMP_LT_OQ); /* cmpltps. or signalling _CMP_LT_OS? */ \
646 vandps(etmp, x, mask); \
647 vsubps(x, x, CPTR_AVX(elfloat_one)); \
648 vandps(mask, mask, CPTR_AVX(elfloat_one)); \
649 vsubps(emm0, emm0, mask); \
650 vaddps(x, x, etmp); \
651 vmulps(z, x, x); \
652 vmovaps(y, CPTR_AVX(elcephes_log_p0)); \
653 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p1)); \
654 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p2)); \
655 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p3)); \
656 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p4)); \
657 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p5)); \
658 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p6)); \
659 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p7)); \
660 vfmadd213ps(y, x, CPTR_AVX(elcephes_log_p8)); \
661 vmulps(y, y, x); \
662 vmulps(y, y, z); \
663 vfmadd231ps(y, emm0, CPTR_AVX(elcephes_log_q1)); \
664 vfnmadd231ps(y, z, CPTR_AVX(elfloat_half)); \
665 vaddps(x, x, y); \
666 vfmadd231ps(x, emm0, CPTR_AVX(elcephes_log_q2)); \
667 vorps(x, x, invalid_mask); }
668
669 // Note: VS Expr changed a lot since ported to Avisynth,
670 // Here we are using their VEX macros for easy port of new sin and cos
671 // however we do not support VEX encoding or FMA3 in xmm register mode
672 // Note: VEX2 cmpltps -> vcmpltps does not work so we uncomment v##op parts as well (comparisons changed in vex)
673 #define VEX1(op, arg1, arg2) \
674 do { \
675 if constexpr(false /*cpuFlags & CPUF_AVX*/) \
676 /*v##op(arg1, arg2)*/; \
677 else \
678 op(arg1, arg2); \
679 } while (0)
680 #define VEX1IMM(op, arg1, arg2, imm) \
681 do { \
682 if constexpr(false /*cpuFlags & CPUF_AVX*/) { \
683 /*v##op(arg1, arg2, imm)*/; \
684 } else if (arg1 == arg2) { \
685 op(arg2, imm); \
686 } else { \
687 movdqa(arg1, arg2); \
688 op(arg1, imm); \
689 } \
690 } while (0)
691 #define VEX2(op, arg1, arg2, arg3) \
692 do { \
693 if constexpr(false /*cpuFlags & CPUF_AVX*/) { \
694 /*v##op(arg1, arg2, arg3)*/; \
695 } else if (arg1 == arg2) { \
696 op(arg2, arg3); \
697 } else if (arg1 != arg3) { \
698 movdqa(arg1, arg2); \
699 op(arg1, arg3); \
700 } else { \
701 XmmReg tmp; \
702 movdqa(tmp, arg2); \
703 op(tmp, arg3); \
704 movdqa(arg1, tmp); \
705 } \
706 } while (0)
707 #define VEX2IMM(op, arg1, arg2, arg3, imm) \
708 do { \
709 if constexpr(false/*cpuFlags & CPUF_AVX*/) { \
710 /*v##op(arg1, arg2, arg3, imm)*/; \
711 } else if (arg1 == arg2) { \
712 op(arg2, arg3, imm); \
713 } else if (arg1 != arg3) { \
714 movdqa(arg1, arg2); \
715 op(arg1, arg3, imm); \
716 } else { \
717 XmmReg tmp; \
718 movdqa(tmp, arg2); \
719 op(tmp, arg3, imm); \
720 movdqa(arg1, tmp); \
721 } \
722 } while (0)
723
724 #if 0
725 // Fast tangent approximation using rational function with 8th order terms
726 float fast_tanf(float x) {
727 // Constants for Pi approximation and range reduction
728 const float polyPI = 3.14159265358979f;
729 const float halfPI = polyPI * 0.5f;
730 // Reduce to [-polyPI, polyPI] range
731 float y = fmodf(x, polyPI);
732 if (y > halfPI) y -= polyPI;
733 else if (y < -halfPI) y += polyPI;
734 // At this point y is in [-polyPI/2, polyPI/2]
735 // For very small angles, return the angle itself
736 // LLVM would add a small epsilon: sign(y)*2^-25 (0x1p-25f)
737 float abs_y = fabsf(y);
738 const float small_limit = 1e-4f;
739 if (abs_y < small_limit) {
740 return y;
741 }
742 // Check proximity to asymptotes
743 const float asympt_limit = 0.8f;
744 float distToAsymptote = halfPI - abs_y;
745 // If very close to ±polyPI/2, use asymptotic approximation
746 if (distToAsymptote < asympt_limit) {
747 // The tangent function approaches 1/distToAsymptote as y approaches +/-polyPI/2
748 // Improved coefficients based on curve fitting to better match std::tan
749 // not needed a 3rd term float asymptotic = 1.0f / (distToAsymptote * (0.9963f + distToAsymptote * (0.3642f + distToAsymptote * 0.0173f)));
750 float asymptotic = 1.0f / (distToAsymptote * (0.97f + distToAsymptote * 0.35f));
751 // Preserve sign
752 return (y < 0) ? -asymptotic : asymptotic;
753 }
754 float y2 = y * y;
755 // Optimized coefficients with terms up to 8th order
756 // Numerator coefficients
757 const float p0 = 1.0f;
758 const float p2 = 0.3333314036f;
759 const float p4 = 0.1333923995f;
760 const float p6 = 0.0533740603f;
761 const float p8 = 0.0245650893f;
762 // Denominator coefficients
763 const float q0 = 1.0f;
764 const float q2 = 0.1333835001f;
765 const float q4 = 0.0089270802f;
766 const float q6 = 0.0005908960f;
767 const float q8 = 0.0000342237f;
768 // Calculate approximation using Horner's method for efficiency
769 float num = y * (p0 + y2 * (p2 + y2 * (p4 + y2 * (p6 + y2 * p8))));
770 float den = q0 + y2 * (q2 + y2 * (q4 + y2 * (q6 + y2 * q8)));
771 return num / den;
772 }
773 // We are good with this, but LLVM does differently:
774 // https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanf.cpp
775 #endif
776
777 // Fast tangent approximation for non-AVX version
778 #define TAN_PS(x0) { \
779 XmmReg x1, x2, x3, x4, x5, x6, x7, x8, x9, x10; \
780 /* Normalize to [-pi, pi] using multiplication and subtraction */ \
781 VEX1(movaps, x1, CPTR(float_invpi)); /* 1/pi */ \
782 VEX2(mulps, x2, x0, x1); /* x / pi */ \
783 /* round to nearest integer */ \
784 VEX1(movaps, x3, CPTR(float_rintf)); /* round to int helper */ \
785 VEX2(addps, x4, x3, x2); /* x/pi + 2^23 */ \
786 VEX2(subps, x4, x4, x3); /* round(x/pi) */ \
787 VEX1(movaps, x5, CPTR(float_pi1)); /* Load pi1 (highest precision part) */ \
788 VEX2(mulps, x6, x4, x5); /* round(x/pi) * pi1 */ \
789 VEX2(subps, x7, x0, x6); /* Remainder after subtracting largest part */ \
790 /* Subtract remaining parts for higher precision */ \
791 VEX1(movaps, x3, CPTR(float_pi2)); \
792 VEX2(mulps, x3, x4, x3); /* round(x/pi) * pi2 */ \
793 VEX2(subps, x7, x7, x3); /* Further refine remainder */ \
794 VEX1(movaps, x3, CPTR(float_pi3)); \
795 VEX2(mulps, x3, x4, x3); /* round(x/pi) * pi3 */ \
796 VEX2(subps, x7, x7, x3); /* Further refine remainder */ \
797 VEX1(movaps, x3, CPTR(float_pi4)); \
798 VEX2(mulps, x3, x4, x3); /* round(x/pi) * pi4 */ \
799 VEX2(subps, x7, x7, x3); /* Final remainder, now x is in range [-pi, pi] */ \
800 /* Check range for symmetry, normalize to [-pi/2, pi/2] */ \
801 VEX2(mulps, x6, x5, CPTR(elfloat_half)); /* halfPI = pi/2 */ \
802 VEX2(cmpltps, x3, x6, x7); /* halfPI < y (cmpgt -> cmplt)*/ \
803 VEX2(subps, x4, x5, x7); /* pi - y */ \
804 /* blend: if y > halfPI, use pi - y */ \
805 VEX1(movaps, x2, x3); \
806 VEX2(andps, x2, x2, x4); \
807 VEX2(andnps, x3, x3, x7); \
808 VEX2(orps, x7, x3, x2); /* now y is <= pi/2 */ \
809 /* Check for y < -halfPI */ \
810 VEX1(movaps, x3, x6); \
811 VEX2(xorps, x3, x3, CPTR(elsignmask)); /* -halfPI */ \
812 VEX2(cmpltps, x2, x7, x3); /* y < -halfPI */ \
813 VEX2(xorps, x4, x5, CPTR(elsignmask)); /* -pi */ \
814 VEX2(subps, x4, x4, x7); /* -pi - y */ \
815 /* blend: if y < -halfPI, use -pi - y */ \
816 VEX1(movaps, x3, x2); \
817 VEX2(andps, x3, x3, x4); \
818 VEX2(andnps, x2, x2, x7); \
819 VEX2(orps, x7, x2, x3); /* now y is in [-pi/2, pi/2] */ \
820 /* Range reduction end */ \
821 \
822 /* Check for small values */ \
823 VEX1(movaps, x2, x7); \
824 VEX2(andps, x2, x2, CPTR(elabsmask)); /* abs_y */ \
825 VEX2(cmpltps, x3, x2, CPTR(float_tan_small_limit)); /* abs_y < small_limit */ \
826 VEX2(andps, x3, x3, x7); /* y * (abs_y < small_limit) */ \
827 \
828 /* Check for asymptotic proximity */ \
829 VEX1(movaps, x4, x6); /* Load halfPI */ \
830 VEX2(subps, x4, x4, x2); /* distToAsymptote = halfPI - abs_y */ \
831 VEX2(cmpltps, x5, x4, CPTR(float_tan_asympt_limit)); /* distToAsymptote < asympt_limit */ \
832 \
833 /* Calculate asymptotic approximation: 1.0f / (distToAsymptote * (0.97f + distToAsymptote * 0.35f)) */ \
834 VEX2(mulps, x8, x4, CPTR(float_tan_asympt_a2)); /* distToAsymptote * 0.35f */ \
835 VEX2(addps, x8, x8, CPTR(float_tan_asympt_a1)); /* 0.97f + distToAsymptote * 0.35f */ \
836 VEX2(mulps, x8, x8, x4); /* distToAsymptote * (0.97f + distToAsymptote * 0.35f) */ \
837 VEX1(movaps, x4, CPTR(elfloat_one)); /* 1.0f */ \
838 VEX2(divps, x9, x4, x8); /* 1.0f / (distToAsymptote * (0.97f + distToAsymptote * 0.35f)) */ \
839 \
840 /* Adjust sign for asymptotic approximation */ \
841 VEX1(movaps, x8, x7); /* original y */ \
842 VEX2(andps, x8, x8, CPTR(elsignmask)); /* sign of y */ \
843 VEX2(xorps, x9, x9, x8); /* apply sign to asymptotic result */ \
844 \
845 /* Calculate rational approximation for regular values */ \
846 VEX1(movaps, x2, x7); \
847 VEX2(mulps, x2, x2, x7); /* y^2 */ \
848 \
849 /* Compute numerator with higher order terms */ \
850 VEX1(movaps, x10, CPTR(float_tan_p8)); /* p8 */ \
851 VEX2(mulps, x10, x10, x2); /* p8*y^2 */ \
852 VEX2(addps, x10, x10, CPTR(float_tan_p6)); /* p6 + p8*y^2 */ \
853 VEX2(mulps, x10, x10, x2); /* y^2 * (p6 + p8*y^2) */ \
854 VEX2(addps, x10, x10, CPTR(float_tan_p4)); /* p4 + y^2 * (p6 + p8*y^2) */ \
855 VEX2(mulps, x10, x10, x2); /* y^2 * (p4 + y^2 * (p6 + p8*y^2)) */ \
856 VEX2(addps, x10, x10, CPTR(float_tan_p2)); /* p2 + y^2 * (p4 + y^2 * (p6 + p8*y^2)) */ \
857 VEX2(mulps, x10, x10, x2); /* y^2 * (p2 + y^2 * (p4 + y^2 * (p6 + p8*y^2))) */ \
858 VEX2(addps, x10, x10, CPTR(float_tan_p0)); /* p0 + y^2 * (p2 + y^2 * (p4 + y^2 * (p6 + p8*y^2))) */ \
859 VEX2(mulps, x10, x10, x7); /* y * (p0 + y^2 * (p2 + y^2 * (p4 + y^2 * (p6 + p8*y^2)))) */ \
860 \
861 /* Compute denominator with higher order terms */ \
862 VEX1(movaps, x8, CPTR(float_tan_q8)); /* q8 */ \
863 VEX2(mulps, x8, x8, x2); /* q8*y^2 */ \
864 VEX2(addps, x8, x8, CPTR(float_tan_q6)); /* q6 + q8*y^2 */ \
865 VEX2(mulps, x8, x8, x2); /* y^2 * (q6 + q8*y^2) */ \
866 VEX2(addps, x8, x8, CPTR(float_tan_q4)); /* q4 + y^2 * (q6 + q8*y^2) */ \
867 VEX2(mulps, x8, x8, x2); /* y^2 * (q4 + y^2 * (q6 + q8*y^2)) */ \
868 VEX2(addps, x8, x8, CPTR(float_tan_q2)); /* q2 + y^2 * (q4 + y^2 * (q6 + q8*y^2)) */ \
869 VEX2(mulps, x8, x8, x2); /* y^2 * (q2 + y^2 * (q4 + y^2 * (q6 + q8*y^2))) */ \
870 VEX2(addps, x8, x8, CPTR(float_tan_q0)); /* q0 + y^2 * (q2 + y^2 * (q4 + y^2 * (q6 + q8*y^2))) */ \
871 \
872 /* Division: numerator/denominator for regular case */ \
873 VEX2(divps, x1, x10, x8); /* Final result: tan(y) = numerator/denominator */ \
874 \
875 /* Select the appropriate result based on conditions */ \
876 /* If close to asymptote, use asymptotic approximation, else use rational approximation */ \
877 VEX1(movaps, x2, x5); /* Load asymptote condition */ \
878 VEX2(andps, x2, x2, x9); /* asymptotic * (distToAsymptote < asympt_limit) */ \
879 VEX2(andnps, x5, x5, x1); /* rational * !(distToAsymptote < asympt_limit) */ \
880 VEX2(orps, x1, x5, x2); /* Blend asymptotic and rational results */ \
881 \
882 /* If very small value, use y itself, otherwise use computed result */ \
883 VEX2(andnps, x5, x3, x1); /* result * !(abs_y < small_limit) */ \
884 VEX2(orps, x0, x3, x5); /* Final result with small value handling */ \
885 }
886
887 // Fast tangent approximation for AVX version
888 #define TAN_PS_AVX(x0) { \
889 YmmReg x1, x2, x3, x4, x5, x6, x7, x8, x9, x10; \
890 /* Normalize to [-pi, pi] using multiplication and subtraction */ \
891 vmovaps(x1, CPTR_AVX(float_invpi)); /* 1/pi */ \
892 vmulps(x2, x0, x1); /* x / pi */ \
893 vroundps(x3, x2, FROUND_TO_NEAREST_INT); /* round(x / pi) */ \
894 vmovaps(x4, CPTR_AVX(float_pi1)); /* Load pi1 (highest precision part) */ \
895 vmulps(x5, x3, x4); /* round(x / pi) * pi1 */ \
896 vsubps(x6, x0, x5); /* Remainder after subtracting largest part */ \
897 /* Subtract remaining parts for higher precision */ \
898 vmovaps(x7, CPTR_AVX(float_pi2)); \
899 vmulps(x7, x3, x7); /* round(x / pi) * pi2 */ \
900 vsubps(x6, x6, x7); /* Further refine remainder */ \
901 vmovaps(x7, CPTR_AVX(float_pi3)); \
902 vmulps(x7, x3, x7); /* round(x / pi) * pi3 */ \
903 vsubps(x6, x6, x7); /* Further refine remainder */ \
904 vmovaps(x7, CPTR_AVX(float_pi4)); \
905 vmulps(x7, x3, x7); /* round(x / pi) * pi4 */ \
906 vsubps(x6, x6, x7); /* Final remainder, now x is in range [-pi, pi] */ \
907 /* Check range for symmetry, normalize to [-pi/2, pi/2] */ \
908 vmulps(x5, x4, CPTR_AVX(elfloat_half)); /* halfPI = pi/2 */ \
909 vcmpps(x7, x6, x5, _CMP_GT_OQ); /* y > halfPI */ \
910 vsubps(x8, x4, x6); /* pi - y */ \
911 vblendvps(x6, x6, x8, x7); /* if (y > halfPI) y = pi - y; */ \
912 /* Check for y < -halfPI */ \
913 vxorps(x7, x5, CPTR_AVX(elsignmask)); /* -halfPI */ \
914 vcmpps(x8, x6, x7, _CMP_LT_OQ); /* y < -halfPI */ \
915 vxorps(x2, x4, CPTR_AVX(elsignmask)); /* -pi */ \
916 vsubps(x2, x2, x6); /* -pi - y */ \
917 vblendvps(x6, x6, x2, x8); /* if (y < -halfPI) y = -pi - y; */ \
918 /* now y is in [-pi/2, pi/2] */ \
919 /* Range reduction end */ \
920 \
921 /* Check for small values */ \
922 vandps(x2, x6, CPTR_AVX(elabsmask)); /* abs_y */ \
923 vcmpps(x3, x2, CPTR_AVX(float_tan_small_limit), _CMP_LT_OQ); /* abs_y < small_limit */ \
924 /* Save small value result for later blending */ \
925 vandps(x7, x6, x3); /* y * (abs_y < small_limit) */ \
926 \
927 /* Check for asymptotic proximity */ \
928 vmovaps(x4, x5); /* Load halfPI */ \
929 vsubps(x4, x4, x2); /* distToAsymptote = halfPI - abs_y */ \
930 vcmpps(x5, x4, CPTR_AVX(float_tan_asympt_limit), _CMP_LT_OQ); /* distToAsymptote < asympt_limit */ \
931 \
932 /* Calculate asymptotic approximation */ \
933 vmulps(x8, x4, CPTR_AVX(float_tan_asympt_a2)); /* distToAsymptote * 0.35f */ \
934 vaddps(x8, x8, CPTR_AVX(float_tan_asympt_a1)); /* 0.97f + distToAsymptote * 0.35f */ \
935 vmulps(x8, x8, x4); /* distToAsymptote * (0.97f + distToAsymptote * 0.35f) */ \
936 vmovaps(x4, CPTR_AVX(elfloat_one)); /* Load 1.0f */ \
937 vdivps(x9, x4, x8); /* 1.0f / (distToAsymptote * (0.97f + distToAsymptote * 0.35f)) */ \
938 \
939 /* Adjust sign for asymptotic approximation */ \
940 vmovaps(x8, x6); /* original y */ \
941 vandps(x8, x8, CPTR_AVX(elsignmask)); /* sign of y */ \
942 vxorps(x9, x9, x8); /* apply sign to asymptotic result */ \
943 \
944 /* Calculate rational approximation for regular values */ \
945 vmulps(x2, x6, x6); /* y^2 */ \
946 \
947 /* Compute numerator with higher order terms using FMA instructions */ \
948 vmovaps(x10, CPTR_AVX(float_tan_p8)); /* p8 */ \
949 vfmadd213ps(x10, x2, CPTR_AVX(float_tan_p6)); /* p6 + p8*y^2 */ \
950 vfmadd213ps(x10, x2, CPTR_AVX(float_tan_p4)); /* p4 + y^2 * (p6 + p8*y^2) */ \
951 vfmadd213ps(x10, x2, CPTR_AVX(float_tan_p2)); /* p2 + y^2 * (p4 + y^2 * (p6 + p8*y^2)) */ \
952 vfmadd213ps(x10, x2, CPTR_AVX(float_tan_p0)); /* p0 + y^2 * (p2 + y^2 * (p4 + y^2 * (p6 + p8*y^2))) */ \
953 vmulps(x10, x10, x6); /* y * (p0 + y^2 * (p2 + y^2 * (p4 + y^2 * (p6 + p8*y^2)))) */ \
954 \
955 /* Compute denominator with higher order terms using FMA instructions */ \
956 vmovaps(x8, CPTR_AVX(float_tan_q8)); /* q8 */ \
957 vfmadd213ps(x8, x2, CPTR_AVX(float_tan_q6)); /* q6 + q8*y^2 */ \
958 vfmadd213ps(x8, x2, CPTR_AVX(float_tan_q4)); /* q4 + y^2 * (q6 + q8*y^2) */ \
959 vfmadd213ps(x8, x2, CPTR_AVX(float_tan_q2)); /* q2 + y^2 * (q4 + y^2 * (q6 + q8*y^2)) */ \
960 vfmadd213ps(x8, x2, CPTR_AVX(float_tan_q0)); /* q0 + y^2 * (q2 + y^2 * (q4 + y^2 * (q6 + q8*y^2))) */ \
961 \
962 /* Division: numerator/denominator */ \
963 vdivps(x1, x10, x8); /* tan(y) = numerator/denominator */ \
964 \
965 /* Select appropriate result based on conditions */ \
966 /* If close to asymptote, use asymptotic approximation, else use rational approximation */ \
967 vblendvps(x1, x1, x9, x5); /* Blend asymptotic and rational results */ \
968 \
969 /* If very small value, use y itself, otherwise use computed result */ \
970 vblendvps(x0, x1, x7, x3); /* Final result with small value handling */ \
971 }
972
973 // atan2: based on https://stackoverflow.com/questions/46210708/atan2-approximation-with-11bits-in-mantissa-on-x86with-sse2-and-armwith-vfpv4?noredirect=1&lq=1
974 #if 0
975 float fast_atan2f(float y, float x)
976 {
977 // max rel err = 3.53486939e-5
978 const float atan2f_rmul = 0.024840285f;
979 const float atan2f_radd = 0.18681418f;
980 const float atan2f_tmul = -0.094097948f;
981 const float atan2f_tadd = -0.33213072f;
982 const float atan2f_halfpi = 1.57079637f;
983 const float atan2f_pi = 3.14159274f;
984
985 float a, r, s, t, c, q, ax, ay, mx, mn;
986 ax = fabsf(x);
987 ay = fabsf(y);
988
989 mx = fmaxf(ay, ax);
990 mn = fminf(ay, ax);
991 a = mn / mx;
992 // Minimax polynomial approximation to atan(a) on [0,1]
993 s = a * a;
994 c = s * a;
995 q = s * s;
996 r = atan2f_rmul * q + atan2f_radd;
997 t = atan2f_tmul * q + atan2f_tadd;
998 r = r * s + t;
999 r = r * c + a;
1000 // Map to full circle
1001 if (ay > ax) r = atan2f_halfpi - r;
1002 if (x < 0) r = atan2f_pi - r;
1003 if (y < 0) r = -r;
1004 return r;
1005 }
1006 #endif
1007
1008 // atan2(0, 0) = 0
1009 // ~speed: "y x atan2" C/SSE2/AVX2:52/480/1000 fps
1010 #define ATAN2_PS(x0 /*y*/, /*x*/x1) { \
1011 XmmReg x2, x3, x4, x5, x6, x7, x8; \
1012 /*VEX1(movaps, x1, x);*/ \
1013 /*VEX1(movaps, x0, y);*/ \
1014 /* Remove sign */ \
1015 VEX1(movaps, x3, CPTR(elabsmask)); \
1016 VEX1(movaps, x2, x1); \
1017 VEX2(andps, x2, x2, x3); /* ax = fabsf (x); */ \
1018 VEX2(andps, x3, x3, x0); /* ay = fabsf (y); */ \
1019 VEX1(movaps, x4, x3); \
1020 VEX2(maxps, x4, x4, x2); /* mx = fmaxf (ay, ax); */ \
1021 VEX1(movaps, x5, x3); \
1022 VEX2(minps, x5, x5, x2); /* fminf (ay, ax); */ \
1023 VEX2(divps, x5, x5, x4); /* a = mn / mx; */ \
1024 VEX1(movaps, x4, x5); \
1025 VEX2(mulps, x4, x4, x5); /* s = a * a; */ \
1026 VEX1(movaps, x6, x4); \
1027 VEX2(mulps, x6, x6, x4); /* q = s * s; */ \
1028 VEX1(movaps, x7, CPTR(float_atan2f_rmul)); \
1029 VEX2(mulps, x7, x7, x6); /* r = atan2f_rmul * q (+ atan2f_radd) */ \
1030 VEX2(addps, x7, x7, CPTR(float_atan2f_radd)); /* r = (atan2f_rmul * q) + atan2f_radd; */ \
1031 VEX2(mulps, x7, x7, x4); /* r = r * s (+ t) */ \
1032 VEX2(mulps, x4, x4, x5); /* c = s * a; */ \
1033 VEX2(mulps, x6, x6, CPTR(float_atan2f_tmul)); /* t = atan2f_tmul * q (+ atan2f_tadd) */ \
1034 VEX2(addps, x6, x6, CPTR(float_atan2f_tadd)); /* t = (atan2f_tmul * q) + atan2f_tadd; */ \
1035 VEX2(addps, x7, x7, x6); /* r = (r * s) + t; */ \
1036 VEX2(mulps, x7, x7, x4); /* r = r * c (+ a) */ \
1037 VEX2(addps, x7, x7, x5); /* r = (r * c) + a */ \
1038 /* Map to full circle */ \
1039 /* if (ay > ax) r = atan2f_halfpi - r; */ \
1040 /* if (x < 0) r = atan2f_pi - r; */ \
1041 /* if (y < 0) r = -r; */ \
1042 /* r = atan2f_halfpi - r */ \
1043 VEX1(movaps, x4, CPTR(float_atan2f_halfpi)); \
1044 VEX2(subps, x4, x4, x7); /* r = atan2f_halfpi - r */ \
1045 VEX2(cmpltps, x2, x2, x3); /* if (ay > ax) */ \
1046 /* blend */ \
1047 VEX1(movaps, x3, x2); \
1048 VEX2(andnps, x3, x3, x7); \
1049 VEX2(andps, x2, x2, x4); \
1050 VEX2(orps, x2, x2, x3); \
1051 /* r = atan2f_pi - r; */ \
1052 VEX1(movaps, x3, CPTR(float_atan2f_pi)); \
1053 VEX2(subps, x3, x3, x2); /* r = atan2f_pi - r */ \
1054 /* if (x < 0) */ \
1055 VEX2(xorps, x4, x4, x4); /* zero */ \
1056 VEX2(cmpltps, x1, x1, x4); /* if (x < 0) */ \
1057 /* blend */ \
1058 VEX1(movaps, x5, x1); \
1059 VEX2(andnps, x5, x5, x2); \
1060 VEX2(andps, x1, x1, x3); \
1061 VEX2(orps, x1, x1, x5); \
1062 /* r = -r; */ \
1063 VEX1(movaps, x2, CPTR(elsignmask)); \
1064 VEX2(subps, x2, x2, x1); /* r = -r */ \
1065 /* if (y < 0) */ \
1066 VEX2(cmpltps, x0, x0, x4); /* if (y < 0) */ \
1067 /* blend */ \
1068 VEX1(movaps, x3, x0); \
1069 VEX2(andnps, x3, x3, x1); \
1070 VEX2(andps, x0, x0, x2); \
1071 VEX2(orps, x0, x0, x3); \
1072 /* extra check when 0,0 given -> convert NaN to 0 */ \
1073 VEX1(movaps, x3, x0); \
1074 VEX2(cmpordps, x3, x3, x3);/* find NaNs. 0: NaN in either. FFFF: both non-Nan */ \
1075 /* mask NaN to zero */ \
1076 VEX2(andps, x0, x0, x3); \
1077 /* return value in y */ \
1078 /* input was "x0 for y */ \
1079 }
1080
1081 #define ATAN2_PS_AVX(x0 /*y*/, x1 /*x*/) { \
1082 YmmReg x2, x3, x4, x5, x6, x7, x8; \
1083 /* Remove sign */ \
1084 /* vmovaps(x1, x); */ \
1085 /* vmovaps(x0, y); */ \
1086 vmovaps(x2, CPTR_AVX(elabsmask)); \
1087 vandps(x8, x1, x2); /* ax = fabsf (x); */ \
1088 vandps(x2, x0, x2); /* ay = fabsf (y); */ \
1089 vmaxps(x4, x8, x2); /* mx = fmaxf (ay, ax); */ \
1090 vminps(x5, x8, x2); /* fminf (ay, ax); */ \
1091 vdivps(x4, x5, x4); /* a = mn / mx; */ \
1092 vmulps(x5, x4, x4); /* s = a * a; */ \
1093 vmulps(x6, x5, x5); /* q = s * s; */ \
1094 vmovaps(x7, CPTR_AVX(float_atan2f_rmul)); \
1095 vmovaps(x3, CPTR_AVX(float_atan2f_radd)); \
1096 vfmadd213ps(x7, x6, CPTR_AVX(float_atan2f_radd)); /* r = atan2f_rmul * q + atan2f_radd; */ \
1097 vmovaps(x3, CPTR_AVX(float_atan2f_tmul)); \
1098 vfmadd213ps(x3, x6, CPTR_AVX(float_atan2f_tadd)); /* t = atan2f_tmul * q + atan2f_tadd */ \
1099 vmulps(x6, x5, x4); \
1100 vfmadd231ps(x3, x5, x7); /* r = r * s + t; */ \
1101 vfmadd213ps(x3, x6, x4); /* r = (r * c) + a */ \
1102 /* Map to full circle */ \
1103 /* if (ay > ax) r = atan2f_halfpi - r; */ \
1104 /* if (x < 0) r = atan2f_pi - r; */ \
1105 /* if (y < 0) r = -r; */ \
1106 /* r = atan2f_pi - r */ \
1107 vmovaps(x4, CPTR_AVX(float_atan2f_halfpi)); \
1108 vsubps(x4, x4, x3); /* r = atan2f_halfpi - r */ \
1109 vcmpps(x2, x8, x2, _CMP_LT_OQ); /*vcmpltps(x2, x8, x2);*/ /* if (ay > ax) */ \
1110 vblendvps(x2, x3, x4, x2); \
1111 vmovaps(x3, CPTR_AVX(float_atan2f_pi)); \
1112 vsubps(x3, x3, x2); /* r = atan2f_pi - r */ \
1113 vxorps(x4, x4, x4); \
1114 vcmpps(x1, x1, x4, _CMP_LT_OQ); /* vcmpltps(x1, x1, x4); */ /* if (x < 0) */ \
1115 vblendvps(x1, x2, x3, x1); \
1116 vmovaps(x2, CPTR_AVX(elsignmask)); \
1117 vxorps(x2, x1, x2); /* r = -r */ \
1118 vcmpps(x0, x0, x4, _CMP_LT_OQ); /* vcmpltps(x0, x0, x4); */ /* if (y < 0) */ \
1119 vblendvps(x0, x1, x2, x0); \
1120 /* extra check when 0,0 given -> convert NaN to 0 */ \
1121 vcmpps(x3, x0, x0, _CMP_ORD_Q); /* vcmpordps, x3, x3, x3);*/ /* find NaNs. 0: NaN in either. FFFF: both non-Nan */ \
1122 /* mask NaN to zero */ \
1123 vandps(x0, x0, x3); \
1124 /* return value in y */ \
1125 /* no need. input was "x0" for y */ \
1126 }
1127
1128 #define SINCOS_PS(issin, y, x) { \
1129 XmmReg t1, sign, t2, t3, t4; \
1130 /* // Remove sign */ \
1131 VEX1(movaps, t1, CPTR(elabsmask)); \
1132 if (issin) { \
1133 VEX1(movaps, sign, t1); \
1134 VEX2(andnps, sign, sign, x); \
1135 } \
1136 else { \
1137 VEX2(pxor, sign, sign, sign); \
1138 } \
1139 VEX2(andps, t1, t1, x); \
1140 /*// Range reduction*/ \
1141 VEX1(movaps, t3, CPTR(float_rintf)); \
1142 VEX2(mulps, t2, t1, CPTR(float_invpi)); \
1143 VEX2(addps, t2, t2, t3); \
1144 VEX1IMM(pslld, t4, t2, 31); \
1145 VEX2(xorps, sign, sign, t4); \
1146 VEX2(subps, t2, t2, t3); \
1147 if constexpr(false /*cpuFlags & CPUF_FMA3*/) { \
1148 vfnmadd231ps(t1, t2, CPTR(float_pi1)); \
1149 vfnmadd231ps(t1, t2, CPTR(float_pi2)); \
1150 vfnmadd231ps(t1, t2, CPTR(float_pi3)); \
1151 vfnmadd231ps(t1, t2, CPTR(float_pi4)); \
1152 } \
1153 else { \
1154 VEX2(mulps, t4, t2, CPTR(float_pi1)); \
1155 VEX2(subps, t1, t1, t4); \
1156 VEX2(mulps, t4, t2, CPTR(float_pi2)); \
1157 VEX2(subps, t1, t1, t4); \
1158 VEX2(mulps, t4, t2, CPTR(float_pi3)); \
1159 VEX2(subps, t1, t1, t4); \
1160 VEX2(mulps, t4, t2, CPTR(float_pi4)); \
1161 VEX2(subps, t1, t1, t4); \
1162 } \
1163 if (issin) { \
1164 /* // Evaluate minimax polynomial for sin(x) in [-pi/2, pi/2] interval */ \
1165 /* // Y <- X + X * X^2 * (C3 + X^2 * (C5 + X^2 * (C7 + X^2 * C9))) */ \
1166 VEX2(mulps, t2, t1, t1); \
1167 if constexpr(false /*cpuFlags & CPUF_FMA3*/) { \
1168 vmovaps(t3, CPTR(float_sinC7)); \
1169 vfmadd231ps(t3, t2, CPTR(float_sinC9)); \
1170 vfmadd213ps(t3, t2, CPTR(float_sinC5)); \
1171 vfmadd213ps(t3, t2, CPTR(float_sinC3)); \
1172 VEX2(mulps, t3, t3, t2); \
1173 vfmadd231ps(t1, t1, t3); \
1174 } \
1175 else { \
1176 VEX2(mulps, t3, t2, CPTR(float_sinC9)); \
1177 VEX2(addps, t3, t3, CPTR(float_sinC7)); \
1178 VEX2(mulps, t3, t3, t2); \
1179 VEX2(addps, t3, t3, CPTR(float_sinC5)); \
1180 VEX2(mulps, t3, t3, t2); \
1181 VEX2(addps, t3, t3, CPTR(float_sinC3)); \
1182 VEX2(mulps, t3, t3, t2); \
1183 VEX2(mulps, t3, t3, t1); \
1184 VEX2(addps, t1, t1, t3); \
1185 } \
1186 } \
1187 else { \
1188 /* // Evaluate minimax polynomial for cos(x) in [-pi/2, pi/2] interval */ \
1189 /* // Y <- 1 + X^2 * (C2 + X^2 * (C4 + X^2 * (C6 + X^2 * C8))) */ \
1190 VEX2(mulps, t2, t1, t1); \
1191 if constexpr(false /*cpuFlags & CPUF_FMA3*/) { \
1192 vmovaps(t1, CPTR(float_cosC6)); \
1193 vfmadd231ps(t1, t2, CPTR(float_cosC8)); \
1194 vfmadd213ps(t1, t2, CPTR(float_cosC4)); \
1195 vfmadd213ps(t1, t2, CPTR(float_cosC2)); \
1196 vfmadd213ps(t1, t2, CPTR(elfloat_one)); \
1197 } \
1198 else { \
1199 VEX2(mulps, t1, t2, CPTR(float_cosC8)); \
1200 VEX2(addps, t1, t1, CPTR(float_cosC6)); \
1201 VEX2(mulps, t1, t1, t2); \
1202 VEX2(addps, t1, t1, CPTR(float_cosC4)); \
1203 VEX2(mulps, t1, t1, t2); \
1204 VEX2(addps, t1, t1, CPTR(float_cosC2)); \
1205 VEX2(mulps, t1, t1, t2); \
1206 VEX2(addps, t1, t1, CPTR(elfloat_one)); \
1207 } \
1208 } \
1209 /*// Apply sign */ \
1210 VEX2(xorps, y, t1, sign); \
1211 }
1212
1213 // y dst x src
1214 #define SINCOS_PS_AVX(issin, y, x) { \
1215 YmmReg t1, sign, t2, t3, t4; \
1216 /* // Remove sign */ \
1217 vmovaps(t1, CPTR_AVX(elabsmask)); \
1218 if (issin) { \
1219 vmovaps(sign, t1); \
1220 vandnps(sign, sign, x); \
1221 } \
1222 else { \
1223 vxorps(sign, sign, sign); \
1224 } \
1225 vandps(t1, t1, x); \
1226 /*// Range reduction*/ \
1227 vmovaps(t3, CPTR_AVX(float_rintf)); \
1228 vmulps(t2, t1, CPTR_AVX(float_invpi)); \
1229 vaddps(t2, t2, t3); \
1230 vpslld(t4, t2, 31); \
1231 vxorps(sign, sign, t4); \
1232 vsubps(t2, t2, t3); \
1233 vfnmadd231ps(t1, t2, CPTR_AVX(float_pi1)); \
1234 vfnmadd231ps(t1, t2, CPTR_AVX(float_pi2)); \
1235 vfnmadd231ps(t1, t2, CPTR_AVX(float_pi3)); \
1236 vfnmadd231ps(t1, t2, CPTR_AVX(float_pi4)); \
1237 if (issin) { \
1238 /* // Evaluate minimax polynomial for sin(x) in [-pi/2, pi/2] interval */ \
1239 /* // Y <- X + X * X^2 * (C3 + X^2 * (C5 + X^2 * (C7 + X^2 * C9))) */ \
1240 vmulps(t2, t1, t1); \
1241 vmovaps(t3, CPTR_AVX(float_sinC7)); \
1242 vfmadd231ps(t3, t2, CPTR_AVX(float_sinC9)); \
1243 vfmadd213ps(t3, t2, CPTR_AVX(float_sinC5)); \
1244 vfmadd213ps(t3, t2, CPTR_AVX(float_sinC3)); \
1245 vmulps(t3, t3, t2); \
1246 vfmadd231ps(t1, t1, t3); \
1247 } \
1248 else { \
1249 /* // Evaluate minimax polynomial for cos(x) in [-pi/2, pi/2] interval */ \
1250 /* // Y <- 1 + X^2 * (C2 + X^2 * (C4 + X^2 * (C6 + X^2 * C8))) */ \
1251 vmulps(t2, t1, t1); \
1252 vmovaps(t1, CPTR_AVX(float_cosC6)); \
1253 vfmadd231ps(t1, t2, CPTR_AVX(float_cosC8)); \
1254 vfmadd213ps(t1, t2, CPTR_AVX(float_cosC4)); \
1255 vfmadd213ps(t1, t2, CPTR_AVX(float_cosC2)); \
1256 vfmadd213ps(t1, t2, CPTR_AVX(elfloat_one)); \
1257 } \
1258 /*// Apply sign */ \
1259 vxorps(y, t1, sign); \
1260 }
1261
1262 // return (x - std::round(x / d)*d);
1263 #define FMOD_PS(x, d) { \
1264 XmmReg aTmp; \
1265 movaps(aTmp, x); \
1266 divps(aTmp, d); \
1267 cvttps2dq(aTmp,aTmp); \
1268 cvtdq2ps(aTmp,aTmp); \
1269 mulps(aTmp, d); \
1270 subps(x, aTmp); }
1271
1272 #define FMOD_PS_AVX(x, d) { \
1273 YmmReg aTmp; \
1274 vdivps(aTmp, x, d); \
1275 vcvttps2dq(aTmp,aTmp); \
1276 vcvtdq2ps(aTmp,aTmp); \
1277 vmulps(aTmp, aTmp, d); \
1278 vsubps(x, x, aTmp); }
1279
1280 struct ExprEval : public jitasm::function<void, ExprEval, uint8_t *, const intptr_t *, intptr_t, intptr_t> {
1281
1282 std::vector<ExprOp> ops;
1283 int numInputs;
1284 int cpuFlags;
1285 int planeheight;
1286 int planewidth;
1287 bool singleMode;
1288 int labelCount; // to have unique label strings
1289
1290 ✗ std::string getLabelCount()
1291 {
1292 ✗ return std::to_string(++labelCount);
1293 }
1294
1295 21 ExprEval(std::vector<ExprOp> &ops, int numInputs, int cpuFlags, int planewidth, int planeheight, bool singleMode) : ops(ops), numInputs(numInputs), cpuFlags(cpuFlags),
1296
1/2
✓ Branch 3 → 4 taken 7 times.
✗ Branch 3 → 5 not taken.
 7 planeheight(planeheight), planewidth(planewidth), singleMode(singleMode), labelCount(0) {}
1297
1298 AVS_FORCEINLINE void doMask(XmmReg &r, Reg &constptr, int _planewidth)
1299 {
1300 9 switch (_planewidth & 3) {
1301
4/48
✗ Branch 13089 → 13090 not taken.
✗ Branch 13089 → 13101 not taken.
✗ Branch 13090 → 13091 not taken.
✗ Branch 13090 → 13101 not taken.
✗ Branch 13129 → 13130 not taken.
✗ Branch 13129 → 13141 not taken.
✗ Branch 13130 → 13131 not taken.
✗ Branch 13130 → 13141 not taken.
✗ Branch 13167 → 13168 not taken.
✗ Branch 13167 → 13179 not taken.
✗ Branch 13168 → 13169 not taken.
✗ Branch 13168 → 13179 not taken.
✗ Branch 13205 → 13206 not taken.
✗ Branch 13205 → 13217 not taken.
✗ Branch 13206 → 13207 not taken.
✗ Branch 13206 → 13217 not taken.
✗ Branch 13240 → 13241 not taken.
✗ Branch 13240 → 13252 not taken.
✗ Branch 13241 → 13242 not taken.
✗ Branch 13241 → 13252 not taken.
✗ Branch 13275 → 13276 not taken.
✗ Branch 13275 → 13287 not taken.
✗ Branch 13276 → 13277 not taken.
✗ Branch 13276 → 13287 not taken.
✗ Branch 25397 → 25398 not taken.
✗ Branch 25397 → 25409 not taken.
✗ Branch 25398 → 25399 not taken.
✗ Branch 25398 → 25409 not taken.
✓ Branch 25432 → 25433 taken 3 times.
✗ Branch 25432 → 25444 not taken.
✓ Branch 25433 → 25434 taken 3 times.
✗ Branch 25433 → 25444 not taken.
✗ Branch 25465 → 25466 not taken.
✗ Branch 25465 → 25477 not taken.
✗ Branch 25466 → 25467 not taken.
✗ Branch 25466 → 25477 not taken.
✗ Branch 25498 → 25499 not taken.
✗ Branch 25498 → 25510 not taken.
✗ Branch 25499 → 25500 not taken.
✗ Branch 25499 → 25510 not taken.
✗ Branch 25527 → 25528 not taken.
✗ Branch 25527 → 25539 not taken.
✗ Branch 25528 → 25529 not taken.
✗ Branch 25528 → 25539 not taken.
✓ Branch 25559 → 25560 taken 3 times.
✗ Branch 25559 → 25571 not taken.
✓ Branch 25560 → 25561 taken 3 times.
✗ Branch 25560 → 25571 not taken.
 6 case 1: andps(r, CPTR(loadmask1000)); break;
1302 ✗ case 2: andps(r, CPTR(loadmask1100)); break;
1303
2/48
✓ Branch 13097 → 13098 taken 3 times.
✗ Branch 13097 → 13105 not taken.
✓ Branch 13098 → 13099 taken 3 times.
✗ Branch 13098 → 13105 not taken.
✗ Branch 13137 → 13138 not taken.
✗ Branch 13137 → 13145 not taken.
✗ Branch 13138 → 13139 not taken.
✗ Branch 13138 → 13145 not taken.
✗ Branch 13175 → 13176 not taken.
✗ Branch 13175 → 13183 not taken.
✗ Branch 13176 → 13177 not taken.
✗ Branch 13176 → 13183 not taken.
✗ Branch 13213 → 13214 not taken.
✗ Branch 13213 → 13221 not taken.
✗ Branch 13214 → 13215 not taken.
✗ Branch 13214 → 13221 not taken.
✗ Branch 13248 → 13249 not taken.
✗ Branch 13248 → 13256 not taken.
✗ Branch 13249 → 13250 not taken.
✗ Branch 13249 → 13256 not taken.
✗ Branch 13283 → 13284 not taken.
✗ Branch 13283 → 13291 not taken.
✗ Branch 13284 → 13285 not taken.
✗ Branch 13284 → 13291 not taken.
✗ Branch 25405 → 25406 not taken.
✗ Branch 25405 → 25413 not taken.
✗ Branch 25406 → 25407 not taken.
✗ Branch 25406 → 25413 not taken.
✗ Branch 25440 → 25441 not taken.
✗ Branch 25440 → 25448 not taken.
✗ Branch 25441 → 25442 not taken.
✗ Branch 25441 → 25448 not taken.
✗ Branch 25473 → 25474 not taken.
✗ Branch 25473 → 25481 not taken.
✗ Branch 25474 → 25475 not taken.
✗ Branch 25474 → 25481 not taken.
✗ Branch 25506 → 25507 not taken.
✗ Branch 25506 → 25514 not taken.
✗ Branch 25507 → 25508 not taken.
✗ Branch 25507 → 25514 not taken.
✗ Branch 25535 → 25536 not taken.
✗ Branch 25535 → 25543 not taken.
✗ Branch 25536 → 25537 not taken.
✗ Branch 25536 → 25543 not taken.
✗ Branch 25567 → 25568 not taken.
✗ Branch 25567 → 25575 not taken.
✗ Branch 25568 → 25569 not taken.
✗ Branch 25568 → 25575 not taken.
 3 case 3: andps(r, CPTR(loadmask1110)); break;
1304 }
1305 9 }
1306
1307 template<bool processSingle, bool maskUnused>
1308 AVS_FORCEINLINE void processingLoop(Reg &regptrs, XmmReg &zero, Reg &constptr, Reg &SpatialY)
1309 {
1310 13 std::list<std::pair<XmmReg, XmmReg>> stack;
1311 13 std::list<XmmReg> stack1;
1312
1313 13 const int pixels_per_cycle = processSingle ? 4 : 8;
1314
1315 13 const bool maskIt = (maskUnused && ((planewidth & 3) != 0));
1316
1317
6/10
✗ Branch 3417 → 27 not taken.
✗ Branch 3417 → 3418 not taken.
✓ Branch 10693 → 4354 taken 29 times.
✓ Branch 10693 → 10694 taken 7 times.
✓ Branch 18938 → 12470 taken 14 times.
✓ Branch 18938 → 18939 taken 3 times.
✗ Branch 24075 → 20685 not taken.
✗ Branch 24075 → 24076 not taken.
✓ Branch 28531 → 25013 taken 12 times.
✓ Branch 28531 → 28532 taken 3 times.
 81 for (const auto &iter : ops) {
1318
5/10
✗ Branch 29 → 30 not taken.
✗ Branch 29 → 42 not taken.
✓ Branch 4356 → 4357 taken 2 times.
✓ Branch 4356 → 4375 taken 27 times.
✓ Branch 12472 → 12473 taken 2 times.
✓ Branch 12472 → 12491 taken 12 times.
✗ Branch 20687 → 20688 not taken.
✗ Branch 20687 → 20700 not taken.
✗ Branch 25015 → 25016 not taken.
✓ Branch 25015 → 25028 taken 12 times.
 55 if (iter.op == opLoadSpatialX) {
1319 if (processSingle) {
1320 ✗ XmmReg r1;
1321 ✗ movd(r1, dword_ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1322 ✗ shufps(r1, r1, 0);
1323 ✗ cvtdq2ps(r1, r1);
1324 ✗ addps(r1, CPTR(spatialX));
1325 ✗ stack1.push_back(r1);
1326 }
1327 else {
1328
4/8
✓ Branch 4357 → 4358 taken 2 times.
✗ Branch 4357 → 10705 not taken.
✓ Branch 4358 → 4359 taken 2 times.
✗ Branch 4358 → 10705 not taken.
✓ Branch 12473 → 12474 taken 2 times.
✗ Branch 12473 → 18950 not taken.
✓ Branch 12474 → 12475 taken 2 times.
✗ Branch 12474 → 18950 not taken.
 4 XmmReg r1, r2;
1329
4/8
✓ Branch 4360 → 4361 taken 2 times.
✗ Branch 4360 → 10697 not taken.
✓ Branch 4361 → 4362 taken 2 times.
✗ Branch 4361 → 10697 not taken.
✓ Branch 12476 → 12477 taken 2 times.
✗ Branch 12476 → 18942 not taken.
✓ Branch 12477 → 12478 taken 2 times.
✗ Branch 12477 → 18942 not taken.
 4 movd(r1, dword_ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1330
4/8
✓ Branch 4362 → 4363 taken 2 times.
✗ Branch 4362 → 10699 not taken.
✓ Branch 4363 → 4364 taken 2 times.
✗ Branch 4363 → 10699 not taken.
✓ Branch 12478 → 12479 taken 2 times.
✗ Branch 12478 → 18944 not taken.
✓ Branch 12479 → 12480 taken 2 times.
✗ Branch 12479 → 18944 not taken.
 4 shufps(r1, r1, 0);
1331
2/4
✓ Branch 4364 → 4365 taken 2 times.
✗ Branch 4364 → 10705 not taken.
✓ Branch 12480 → 12481 taken 2 times.
✗ Branch 12480 → 18950 not taken.
 4 cvtdq2ps(r1, r1);
1332
2/4
✓ Branch 4365 → 4366 taken 2 times.
✗ Branch 4365 → 10705 not taken.
✓ Branch 12481 → 12482 taken 2 times.
✗ Branch 12481 → 18950 not taken.
 4 movaps(r2, r1);
1333
4/8
✓ Branch 4367 → 4368 taken 2 times.
✗ Branch 4367 → 10700 not taken.
✓ Branch 4368 → 4369 taken 2 times.
✗ Branch 4368 → 10700 not taken.
✓ Branch 12483 → 12484 taken 2 times.
✗ Branch 12483 → 18945 not taken.
✓ Branch 12484 → 12485 taken 2 times.
✗ Branch 12484 → 18945 not taken.
 4 addps(r1, CPTR(spatialX));
1334
4/8
✓ Branch 4370 → 4371 taken 2 times.
✗ Branch 4370 → 10702 not taken.
✓ Branch 4371 → 4372 taken 2 times.
✗ Branch 4371 → 10702 not taken.
✓ Branch 12486 → 12487 taken 2 times.
✗ Branch 12486 → 18947 not taken.
✓ Branch 12487 → 12488 taken 2 times.
✗ Branch 12487 → 18947 not taken.
 4 addps(r2, CPTR(spatialX2));
1335
4/8
✓ Branch 4372 → 4373 taken 2 times.
✗ Branch 4372 → 10704 not taken.
✓ Branch 4373 → 4374 taken 2 times.
✗ Branch 4373 → 10704 not taken.
✓ Branch 12488 → 12489 taken 2 times.
✗ Branch 12488 → 18949 not taken.
✓ Branch 12489 → 12490 taken 2 times.
✗ Branch 12489 → 18949 not taken.
 4 stack.push_back(std::make_pair(r1, r2));
1336 }
1337 }
1338
5/10
✗ Branch 42 → 43 not taken.
✗ Branch 42 → 50 not taken.
✓ Branch 4375 → 4376 taken 2 times.
✓ Branch 4375 → 4386 taken 25 times.
✓ Branch 12491 → 12492 taken 2 times.
✓ Branch 12491 → 12502 taken 10 times.
✗ Branch 20700 → 20701 not taken.
✗ Branch 20700 → 20708 not taken.
✗ Branch 25028 → 25029 not taken.
✓ Branch 25028 → 25036 taken 12 times.
 51 else if (iter.op == opLoadSpatialY) {
1339 if (processSingle) {
1340 ✗ XmmReg r1;
1341 ✗ movd(r1, SpatialY);
1342 ✗ shufps(r1, r1, 0);
1343 ✗ cvtdq2ps(r1, r1);
1344 ✗ stack1.push_back(r1);
1345 }
1346 else {
1347
4/8
✓ Branch 4376 → 4377 taken 2 times.
✗ Branch 4376 → 10708 not taken.
✓ Branch 4377 → 4378 taken 2 times.
✗ Branch 4377 → 10708 not taken.
✓ Branch 12492 → 12493 taken 2 times.
✗ Branch 12492 → 18953 not taken.
✓ Branch 12493 → 12494 taken 2 times.
✗ Branch 12493 → 18953 not taken.
 4 XmmReg r1, r2;
1348
2/4
✓ Branch 4378 → 4379 taken 2 times.
✗ Branch 4378 → 10708 not taken.
✓ Branch 12494 → 12495 taken 2 times.
✗ Branch 12494 → 18953 not taken.
 4 movd(r1, SpatialY);
1349
4/8
✓ Branch 4379 → 4380 taken 2 times.
✗ Branch 4379 → 10706 not taken.
✓ Branch 4380 → 4381 taken 2 times.
✗ Branch 4380 → 10706 not taken.
✓ Branch 12495 → 12496 taken 2 times.
✗ Branch 12495 → 18951 not taken.
✓ Branch 12496 → 12497 taken 2 times.
✗ Branch 12496 → 18951 not taken.
 4 shufps(r1, r1, 0);
1350
2/4
✓ Branch 4381 → 4382 taken 2 times.
✗ Branch 4381 → 10708 not taken.
✓ Branch 12497 → 12498 taken 2 times.
✗ Branch 12497 → 18953 not taken.
 4 cvtdq2ps(r1, r1);
1351
2/4
✓ Branch 4382 → 4383 taken 2 times.
✗ Branch 4382 → 10708 not taken.
✓ Branch 12498 → 12499 taken 2 times.
✗ Branch 12498 → 18953 not taken.
 4 movaps(r2, r1);
1352
4/8
✓ Branch 4383 → 4384 taken 2 times.
✗ Branch 4383 → 10707 not taken.
✓ Branch 4384 → 4385 taken 2 times.
✗ Branch 4384 → 10707 not taken.
✓ Branch 12499 → 12500 taken 2 times.
✗ Branch 12499 → 18952 not taken.
✓ Branch 12500 → 12501 taken 2 times.
✗ Branch 12500 → 18952 not taken.
 4 stack.push_back(std::make_pair(r1, r2));
1353 }
1354 }
1355
3/10
✗ Branch 50 → 51 not taken.
✗ Branch 50 → 59 not taken.
✗ Branch 4386 → 4387 not taken.
✓ Branch 4386 → 4398 taken 25 times.
✗ Branch 12502 → 12503 not taken.
✓ Branch 12502 → 12514 taken 10 times.
✗ Branch 20708 → 20709 not taken.
✗ Branch 20708 → 20717 not taken.
✗ Branch 25036 → 25037 not taken.
✓ Branch 25036 → 25045 taken 12 times.
 47 else if (iter.op == opLoadInternalVar) {
1356 if (processSingle) {
1357 ✗ XmmReg r1;
1358 ✗ movd(r1, dword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INTERNAL_VARIABLES)]);
1359 ✗ shufps(r1, r1, 0);
1360 ✗ stack1.push_back(r1);
1361 }
1362 else {
1363 ✗ XmmReg r1, r2;
1364 ✗ movd(r1, dword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INTERNAL_VARIABLES)]);
1365 ✗ shufps(r1, r1, 0);
1366 ✗ movaps(r2, r1);
1367 ✗ stack.push_back(std::make_pair(r1, r2));
1368 }
1369 }
1370
3/10
✗ Branch 59 → 60 not taken.
✗ Branch 59 → 68 not taken.
✗ Branch 4398 → 4399 not taken.
✓ Branch 4398 → 4410 taken 25 times.
✗ Branch 12514 → 12515 not taken.
✓ Branch 12514 → 12526 taken 10 times.
✗ Branch 20717 → 20718 not taken.
✗ Branch 20717 → 20726 not taken.
✗ Branch 25045 → 25046 not taken.
✓ Branch 25045 → 25054 taken 12 times.
 47 else if (iter.op == opLoadFramePropVar) {
1371 if (processSingle) {
1372 ✗ XmmReg r1;
1373 ✗ movd(r1, dword_ptr[regptrs + sizeof(void*) * (iter.e.ival + RWPTR_START_OF_INTERNAL_FRAMEPROP_VARIABLES)]);
1374 ✗ shufps(r1, r1, 0);
1375 ✗ stack1.push_back(r1);
1376 }
1377 else {
1378 ✗ XmmReg r1, r2;
1379 ✗ movd(r1, dword_ptr[regptrs + sizeof(void*) * (iter.e.ival + RWPTR_START_OF_INTERNAL_FRAMEPROP_VARIABLES)]);
1380 ✗ shufps(r1, r1, 0);
1381 ✗ movaps(r2, r1);
1382 ✗ stack.push_back(std::make_pair(r1, r2));
1383 }
1384 }
1385
9/30
✗ Branch 68 → 69 not taken.
✗ Branch 68 → 71 not taken.
✗ Branch 69 → 70 not taken.
✗ Branch 69 → 71 not taken.
✗ Branch 70 → 71 not taken.
✗ Branch 70 → 396 not taken.
✓ Branch 4410 → 4411 taken 25 times.
✗ Branch 4410 → 4413 not taken.
✓ Branch 4411 → 4412 taken 25 times.
✗ Branch 4411 → 4413 not taken.
✗ Branch 4412 → 4413 not taken.
✓ Branch 4412 → 4954 taken 25 times.
✓ Branch 12526 → 12527 taken 10 times.
✗ Branch 12526 → 12529 not taken.
✓ Branch 12527 → 12528 taken 10 times.
✗ Branch 12527 → 12529 not taken.
✗ Branch 12528 → 12529 not taken.
✓ Branch 12528 → 13070 taken 10 times.
✗ Branch 20726 → 20727 not taken.
✗ Branch 20726 → 20729 not taken.
✗ Branch 20727 → 20728 not taken.
✗ Branch 20727 → 20729 not taken.
✗ Branch 20728 → 20729 not taken.
✗ Branch 20728 → 21054 not taken.
✓ Branch 25054 → 25055 taken 12 times.
✗ Branch 25054 → 25057 not taken.
✓ Branch 25055 → 25056 taken 12 times.
✗ Branch 25055 → 25057 not taken.
✗ Branch 25056 → 25057 not taken.
✓ Branch 25056 → 25382 taken 12 times.
 47 else if (iter.op == opLoadRelSrc8 || iter.op == opLoadRelSrc16 || iter.op == opLoadRelSrcF32) {
1386 // either dx or dy is nonzero
1387 // common part follows for single 4 pixels/cycle and dual 8 pixels/cycle
1388 ✗ Reg newx;
1389 ✗ if (iter.dx != 0) {
1390 ✗ mov(newx, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]); // original base
1391 ✗ add(newx, iter.dx); // new base
1392 }
1393
1394 ✗ Reg a;
1395 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]); // current pixel group of current line
1396 // adjust read pointer vertically for nonzero dy, keep 0..height-1 limits
1397 ✗ if (iter.dy < 0) {
1398 // Read from above
1399 ✗ Reg dy, sy;
1400 ✗ mov(sy, SpatialY);
1401 ✗ mov(dy, -iter.dy); // dy = -dy;
1402 ✗ cmp(dy, sy);
1403 ✗ cmovg(dy, sy); // mov if greater: if (dy > SpatialY) dy = SpatialY;
1404 #ifdef JITASM64
1405 ✗ imul(dy, qword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_STRIDES)]); // dy * stride
1406 #else
1407 imul(dy, dword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_STRIDES)]); // dy * stride
1408 #endif
1409 ✗ sub(a, dy); // a -= dy * stride
1410 }
1411 ✗ else if (iter.dy > 0) {
1412 // Read from bottom
1413 ✗ Reg dy, sy;
1414 ✗ mov(sy, planeheight - 1);
1415 ✗ sub(sy, SpatialY);
1416 ✗ mov(dy, iter.dy);
1417 ✗ cmp(dy, sy);
1418 ✗ cmovg(dy, sy); // mov if greater: if (dy > (planeheight - 1) - SpatialY) dy = SpatialY;
1419 #ifdef JITASM64
1420 ✗ imul(dy, qword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_STRIDES)]); // dy * stride
1421 #else
1422 imul(dy, dword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_STRIDES)]); // dy * stride
1423 #endif
1424 ✗ add(a, dy); // a += dy * stride
1425 }
1426
1427 // dy shift is done already. newx holds xcounter + dx
1428 // Cases:
1429 // ReadBefore: xcounter + dx < 0 (only when dx < 0):
1430 // FullReadBefore: dx <= pixels_per_cycle: clone leftmost pixel to each pixel posision in the group
1431 // PartialReadBefore: pixels_per_cycle < dx < 0: close leftmost pixel to -dx positions
1432 // NormalRead: 0 <= xcounter + dx < planewidth - (pixels_per_cycle - 1) (can read whole pixel group)
1433 // OverRead:
1434 // PartialOverRead when pixel at (planewidth-1) is current read position
1435 // PartialOverRead when pixel at (planewidth-1) is after current read position
1436 // FullOverRead: clone pixel at (planewidth-1) to each pixel posision in the group
1437 if (processSingle) {
1438 // LoadRel8/16/32, single register mode 1x4 pixels
1439
1440 // Use getLabelCount: names should be unique across multiple calls to processingLoop
1441 ✗ std::string LabelNeg = "neg" + getLabelCount();
1442 ✗ std::string LabelOver = "over" + getLabelCount();
1443 ✗ std::string LabelEnd = "end" + getLabelCount();
1444
1445 ✗ XmmReg r1;
1446
1447 ✗ if (iter.dx < 0) { // Optim: read from left is possible only for dx<0 case
1448 ✗ cmp(newx, 0);
1449 ✗ jl(LabelNeg); // newx < 0, read (partially or fully) from before the leftmost pixel
1450 }
1451 ✗ if (iter.dx != 0) { // Optim: read after rightmost pixel is possible only for dx>0 case
1452 // Also check for dx<0, because of possible memory overread
1453 // e.g.: planewidth = 64, dx = -1, 16 bit pixels, 16 bytes/cycle, reading from offsets -1(0), 15, 31, 47, then 63
1454 // When we read 16 bytes from offset 63, we are overaddressing the 64 byte scanline,
1455 // which may give access violation when pointer is in the most bottom line.
1456 ✗ cmp(newx, planewidth - (pixels_per_cycle - 1)); // read (partially of fully) after the rightmost pixel
1457 ✗ jge(LabelOver);
1458 }
1459
1460 // It's safe to read the whole pixel group
1461 int offset;
1462 ✗ if (iter.op == opLoadRelSrc8)
1463 ✗ offset = iter.dx;
1464 ✗ else if (iter.op == opLoadRelSrc16)
1465 ✗ offset = iter.dx * sizeof(uint16_t);
1466 ✗ else if (iter.op == opLoadRelSrcF32)
1467 ✗ offset = iter.dx * sizeof(float);
1468
1469 ✗ if (iter.op == opLoadRelSrc8) {
1470 ✗ movd(r1, dword_ptr[a + offset]); // 4 pixels, 4 bytes
1471 ✗ punpcklbw(r1, zero);
1472 ✗ punpcklwd(r1, zero);
1473 ✗ cvtdq2ps(r1, r1);
1474 }
1475 ✗ else if (iter.op == opLoadRelSrc16) {
1476 ✗ movq(r1, mmword_ptr[a + offset]); // 4 pixels, 8 bytes
1477 ✗ punpcklwd(r1, zero);
1478 ✗ cvtdq2ps(r1, r1);
1479 }
1480 ✗ else if (iter.op == opLoadRelSrcF32) {
1481 ✗ if (iter.dx % 4 == 0)
1482 ✗ movdqa(r1, xmmword_ptr[a + offset]); // 4 pixels, 16 bytes aligned
1483 else
1484 ✗ movdqu(r1, xmmword_ptr[a + offset]); // 4 pixels, 16 bytes unaligned
1485 }
1486 ✗ if (iter.dx != 0) {
1487 ✗ jmp(LabelEnd); // generate jump only when over/negative branches exist
1488 }
1489
1490 ✗ if (iter.dx != 0) {
1491 ✗ L(LabelOver);
1492 ✗ std::string PartialOverread = "PartialOverread" + getLabelCount();
1493 ✗ std::string NoFullOverReadFromNewX = "NoFullOverReadFromNewX" + getLabelCount();
1494 ✗ std::string labelDoOver = "DoOver" + getLabelCount();
1495
1496 ✗ if (iter.dx > 0) { // FullOverRead possible only when dx>0
1497 ✗ cmp(newx, planewidth);
1498 ✗ jl(PartialOverread); // if newx < planewidth ->
1499
1500 // case: FullOver
1501 // even the first pixel to read is beyond the end of line
1502 // We have to clone the rightmost pixel from (planewidth-1)
1503 ✗ if (iter.op == opLoadRelSrc8) {
1504 ✗ sub(a, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1505 ✗ add(a, planewidth - 1);
1506 // reuse newx
1507 ✗ movzx(newx, byte_ptr[a]);
1508 ✗ movd(r1, newx);
1509 ✗ punpcklbw(r1, zero); // words
1510 ✗ pshufb(r1, CPTR(elShuffleForRight6)); // duplicate last word to all
1511
1512 ✗ punpcklwd(r1, zero);
1513 ✗ cvtdq2ps(r1, r1);
1514 }
1515 ✗ else if (iter.op == opLoadRelSrc16) {
1516 ✗ Reg tmp;
1517 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1518 ✗ shl(tmp, 1); // for 16 bit 2*xcounter
1519 ✗ sub(a, tmp);
1520 ✗ add(a, (planewidth - 1) * 2);
1521 // reuse newx
1522 ✗ movzx(newx, word_ptr[a]);
1523 ✗ movd(r1, newx);
1524 ✗ pshufb(r1, CPTR(elShuffleForRight6)); // duplicate last word to all
1525
1526 ✗ punpcklwd(r1, zero);
1527 ✗ cvtdq2ps(r1, r1);
1528 }
1529 ✗ else if (iter.op == opLoadRelSrcF32) {
1530 ✗ Reg tmp;
1531 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1532 ✗ shl(tmp, 2); // for 32 bit 4*xcounter
1533 ✗ sub(a, tmp);
1534 ✗ add(a, (planewidth - 1) * 4);
1535 ✗ movd(r1, dword_ptr[a]);
1536 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6));
1537 }
1538 ✗ jmp(LabelEnd);
1539 } // full OverRead path, needed when iter.dx>0
1540
1541 // case: Partial overread
1542 // read the block, then clone the last valid pixel from position (planewidth-1)
1543 // problem: newx is not aligned
1544 ✗ L(PartialOverread);
1545
1546 // planewidth == 14 dx=7 newx = 0+7=7, newx>=planewidth-7, then not newx>=planewidth => newx = 13 => planewidth-newx = 1
1547 // sample 1: newx is in different segment than planewidth-1
1548 // [xcounter]
1549 // [a] [newx] [pw-1]
1550 // V V V
1551 // 0 1 2 3 4 5 6 7 8 9 A B C D e f g h i j k l
1552 // P Q R S T U V w need this
1553 // 0 1 2 3 4 5 6 7 we can read this
1554 // P Q R S T U V w last pixel is beyond
1555 // P Q R S T U V V need this
1556 // sample 2: newx is in the same segment than planewidth-1
1557 // planewidth == 13 dx=3
1558 // [xcounter] [newx]
1559 // [a][newx][pw-1]
1560 // V V V
1561 // 0 1 2 3 4 5 6 7 8 9 A B C d e f g h i j
1562 // P Q x x x x x x need this
1563 // P Q Q Q Q Q Q Q duplicated the last valid pixel
1564 // 0 1 2 3 4 5 6 7 we can read this
1565 // when newx and (planewidth-1) are in different segments then we read from newx
1566 ✗ Reg tmp;
1567 ✗ mov(tmp, newx);
1568 ✗ and_(tmp, ~(pixels_per_cycle - 1));
1569 ✗ cmp(tmp, (planewidth & ~(pixels_per_cycle - 1)));
1570 ✗ jle(NoFullOverReadFromNewX); // jump if (newx and ~0x07) < (planewidth & ~0x07) (in another segment)
1571
1572 // read from current (last) pointer,
1573 ✗ if (iter.op == opLoadRelSrc8 || iter.op == opLoadRelSrc16) {
1574 ✗ if (iter.op == opLoadRelSrc8) {
1575 ✗ movd(r1, dword_ptr[a]); // 4 pixels, 4 bytes
1576 ✗ punpcklbw(r1, zero); // words
1577 }
1578 else { // opLoadRel16
1579 ✗ movq(r1, mmword_ptr[a]); // 8 pixels, 16 bytes, here still aligned
1580 }
1581 /*
1582 psrldq(r1, ((planewidth - 1) & (pixels_per_cycle - 1)) * sizeof(uint16_t)); // Shift right by (planewidth - 1) & 7 to lose low words
1583 sub(newx, planewidth - (pixels_per_cycle - 1)); // find out shuffle pointer -1, ... -7 -> 6 ... 0
1584 shl(newx, 4); // *16 for shuffle table
1585 // LabelDoOver copied here
1586 // reuse a : Reg shuffleTable;
1587 lea(a, CPTR(elShuffleForRight0)); // ptr for word shuffle
1588 pshufb(r1, xmmword_ptr[a + newx]);
1589 */
1590 ✗ punpcklwd(r1, zero);
1591 ✗ cvtdq2ps(r1, r1);
1592 //jmp(LabelEnd);
1593 //jmp(labelDoOver);
1594 }
1595 ✗ else if (iter.op == opLoadRelSrcF32) {
1596 // omg it's complicated
1597 ✗ movdqa(r1, xmmword_ptr[a]); // 4 pixels, 16 bytes, here still aligned
1598 }
1599 ✗ int bytes_to_shift = ((planewidth - 1) & (pixels_per_cycle - 1)) * sizeof(float);
1600 ✗ if (bytes_to_shift > 0) {
1601 ✗ psrldq(r1, bytes_to_shift);
1602 ✗ switch (bytes_to_shift) { // 4, 8, 12
1603 ✗ case 4:
1604 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (2 << 4) | (2 << 6));
1605 ✗ break;
1606 ✗ case 8:
1607 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (1 << 4) | (1 << 6));
1608 ✗ break;
1609 ✗ case 12:
1610 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6));
1611 ✗ break;
1612 }
1613 }
1614 ✗ jmp(LabelEnd);
1615 //}
1616
1617 ✗ L(NoFullOverReadFromNewX);
1618 // read from newx
1619 ✗ if (iter.op == opLoadRelSrc8) {
1620 ✗ sub(a, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]); // back x counter bytes to the beginning
1621 ✗ add(a, newx); // new position
1622 ✗ movd(r1, dword_ptr[a]); // 4 pixels, 4 bytes
1623 ✗ punpcklbw(r1, zero); // words
1624 /*
1625 // no shift here, just duplicate appropriate pixel into the high ones
1626 mov(newx, (6 - ((planewidth - iter.dx - 1) & (pixels_per_cycle - 1))) << 4); // find out shuffle pointer
1627 // shuffle by the pattern, table offset in newx, and finalizes
1628 // here r1 contains words
1629 // todo direct load
1630 Reg shuffleTable;
1631 lea(shuffleTable, CPTR(elShuffleForRight4)); // for dual: elShuffleForRight0
1632 add(shuffleTable, newx);
1633 pshufb(r1, xmmword_ptr[shuffleTable]);
1634 */
1635 ✗ punpcklwd(r1, zero);
1636 ✗ cvtdq2ps(r1, r1);
1637 }
1638 ✗ else if (iter.op == opLoadRelSrc16) {
1639 //a = a - 2 * xcounter + 2 * newx;
1640 ✗ sub(newx, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1641 ✗ shl(newx, 1);
1642 ✗ add(a, newx);
1643 ✗ movq(r1, mmword_ptr[a]); // 4 pixels, 8 bytes
1644 // no shift here, just duplicate appropriate pixel into the high ones
1645 /*
1646 // reuse newx
1647 mov(newx, (6 - ((planewidth - iter.dx - 1) & (pixels_per_cycle - 1))) << 4); // find out shuffle pointer
1648 // ((planewidth - iter.dx - 1) & (pixels_per_cycle - 1)) keep
1649 // 0 ShuffleForRight6 keep word #0, spread it to 1..7
1650 // 1 ShuffleForRight5 keep word #0..1, spread it to 2..7
1651 //
1652 // 6 ShuffleForRight0 keep word #0..6, spread it to 7..7
1653 // continues on labelDoOver
1654 // todo direct load
1655 Reg shuffleTable;
1656 lea(shuffleTable, CPTR(elShuffleForRight4)); // for dual: elShuffleForRight0
1657 add(shuffleTable, newx);
1658 pshufb(r1, xmmword_ptr[shuffleTable]);
1659 */
1660 ✗ punpcklwd(r1, zero);
1661 ✗ cvtdq2ps(r1, r1);
1662 }
1663 ✗ else if (iter.op == opLoadRelSrcF32) {
1664 //a = a - 4 * xcounter + 4 * newx;
1665 ✗ sub(newx, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1666 ✗ shl(newx, 2);
1667 ✗ add(a, newx);
1668
1669
1670 // no real shift here, just duplicate appropriate pixel into the high ones. But we have two registers
1671 ✗ movdqu(r1, xmmword_ptr[a]); // 4 pixels, 16 bytes, no need for upper 4 pixels
1672 }
1673 // we have 4 floats here in r1, common part
1674 ✗ int what = ((planewidth - iter.dx - 1) & (pixels_per_cycle - 1));
1675 // what
1676 // 0 ShuffleForRight2_32 keep r1 dword #0 , spread it to 1..3, then spread r1.3 to r2 (ShuffleForRight2_32(r2,r1)
1677 // 1 ShuffleForRight1_32 keep r1 dword #0..1, spread it to 2..3, then spread r1.3 to r2
1678 // 2 ShuffleForRight0_32 keep r1 dword #0..2, spread it to 3 , then spread r1.3 to r2
1679 // 3 keep r1 dword #0..3, , then spread r1.3 to r2
1680 // 4 keep r1 dword #0..3, , keep r2 dword #0 , spread it to 1..3
1681 // 5 keep r1 dword #0..3, , keep r2 dword #0..1, spread it to 2..3
1682 // 6 keep r1 dword #0..3, , keep r2 dword #0..2, spread it to 3
1683
1684 ✗ switch (what) {
1685 ✗ case 0:
1686 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6)); // fill 3 upper dwords of r1 from r1.0
1687 ✗ break;
1688 ✗ case 1:
1689 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (1 << 4) | (1 << 6)); // fill 2 upper dwords of r1 from r1.1
1690 ✗ break;
1691 ✗ case 2:
1692 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (2 << 4) | (2 << 6)); // fill 1 upper dwords of r1 from r1.2
1693 ✗ break;
1694 }
1695 // continues on labelEnd
1696 //}
1697 ✗ if (iter.dx < 0)
1698 ✗ jmp(LabelEnd);
1699 ✗ } // over: iter.dx != 0
1700 ✗ if (iter.dx < 0) {
1701 ✗ L(LabelNeg);
1702 // read from negative area on the left side, read exactly from 0th, then shift
1703 // When reading from negative x coordinates we read exactly from 0th, then shift and duplicate
1704 // For extreme minus offsets we duplicate 0th (leftmost) pixel to each position
1705 // example: dx = -1
1706 // -1 0 1 2 3 4 5 6 7
1707 // A A B C D E F G we need this
1708 // A B C D E F G H read [0]
1709 // 0 A B C D E F G H shift
1710 // A A B C D E F G H duplicate by shuffle
1711 ✗ if (iter.op == opLoadRelSrc8 || iter.op == opLoadRelSrc16) {
1712 ✗ if (iter.op == opLoadRelSrc8) {
1713 ✗ sub(a, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]); // go back to the beginning
1714 ✗ movd(r1, dword_ptr[a]); // 8 pixels, 8 bytes
1715 ✗ punpcklbw(r1, zero); // bytes to words
1716 }
1717 ✗ else if (iter.op == opLoadRelSrc16) {
1718 // go back to the beginning, in 16 bit, *2
1719 ✗ Reg tmp;
1720 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1721 ✗ shl(tmp, 1); // for 16 bit 2*xcounter
1722 ✗ sub(a, tmp);
1723 ✗ movq(r1, mmword_ptr[a]); // 8 pixels, 16 bytes
1724 }
1725 ✗ punpcklwd(r1, zero);
1726 ✗ cvtdq2ps(r1, r1);
1727 ✗ }
1728 ✗ else if (iter.op == opLoadRelSrcF32) {
1729 ✗ Reg tmp;
1730 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1731 ✗ shl(tmp, 2); // *4
1732 ✗ sub(a, tmp);
1733 ✗ movdqa(r1, xmmword_ptr[a]); // 4 pixels, 16 bytes aligned
1734 }
1735 ✗ std::string PartialReadBefore = "PartialReadBefore" + getLabelCount();
1736
1737 ✗ cmp(newx, -pixels_per_cycle);
1738 ✗ jg(PartialReadBefore);
1739 // FullReadBefore: newx <= -pixels_per_cycle, clone 0th (leftmost) pixel to all
1740 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6));
1741 ✗ jmp(LabelEnd);
1742
1743 ✗ L(PartialReadBefore);
1744 // -pixels_per_cycle < newx < 0
1745 ✗ int bytes_to_shift = sizeof(float) * min(pixels_per_cycle - 1, (-iter.dx) & (pixels_per_cycle - 1));
1746 // shift bytes
1747 // 4 r1 << 4 shuffle r1.1 to r1.0-0
1748 // 8 r1 << 8 shuffle r1.2 to r1.0-1
1749 // 12 r1 << 12 shuffle r1.3 to r1.0-2
1750 ✗ pslldq(r1, bytes_to_shift); // todo: shift + shuffle = single shuffle
1751
1752 ✗ switch (bytes_to_shift) { // 4, 8, 12
1753 ✗ case 4:
1754 ✗ pshufd(r1, r1, (1 << 0) | (1 << 2) | (2 << 4) | (3 << 6)); // elShuffleForLeft0_32 // shuffle r1.1 to r1.0-0
1755 ✗ break;
1756 ✗ case 8:
1757 ✗ pshufd(r1, r1, (2 << 0) | (2 << 2) | (2 << 4) | (3 << 6)); // elShuffleForLeft1_32 // shuffle r1.2 to r1.0-1
1758 ✗ break;
1759 ✗ case 12:
1760 ✗ pshufd(r1, r1, (3 << 0) | (3 << 2) | (3 << 4) | (3 << 6)); // elShuffleForLeft2_32 // shuffle r1.3 to r1.0-2
1761 ✗ break;
1762 }
1763 ✗ } // negative
1764 ✗ L(LabelEnd);
1765 ✗ stack1.push_back(r1);
1766 ✗ } // end of single Relative Mode
1767 else {
1768 // LoadRel8/16/32, dual register mode 2x4 pixels
1769
1770 // Use getLabelCount: names should be unique across multiple calls to processingLoop
1771 ✗ std::string LabelNeg = "neg" + getLabelCount();
1772 ✗ std::string LabelOver = "over" + getLabelCount();
1773 ✗ std::string LabelEnd = "end" + getLabelCount();
1774
1775 ✗ XmmReg r1, r2;
1776
1777 // damn, when the order of the two comparisons is exchanged, bad code is generated for dx=-1 (expr=x[-1]).
1778 // jitasm cannot guess the proper register for 'a', it uses register 'xcounter' instead
1779 // maybe the jump order has to match the label order?
1780 // good: LabelOver, LabelNeg. Bad: LabelNeg, LabelOver
1781 // But it's not true. Now x[-2] fails for 32 bit clip
1782 ✗ if (iter.dx < 0) { // Optim: read from left is possible only for dx<0 case
1783 ✗ cmp(newx, 0);
1784 ✗ jl(LabelNeg); // newx < 0, read (partially or fully) from before the leftmost pixel
1785 }
1786 ✗ if (iter.dx != 0) {
1787 // Also check for dx<0, because of possible memory overread
1788 // e.g.: planewidth = 64, dx = -1, 16 bit pixels, 16 bytes/cycle, reading from offsets -1(0), 15, 31, 47, then 63
1789 // When we read 16 bytes from offset 63, we are overaddressing the 64 byte scanline,
1790 // which may give access violation when pointer is in the most bottom line.
1791 ✗ cmp(newx, planewidth - (pixels_per_cycle - 1)); // read (partially of fully) after the rightmost pixel
1792 ✗ jge(LabelOver);
1793 }
1794 /*
1795 if (iter.dx < 0) { // Optim: read from left is possible only for dx<0 case
1796 cmp(newx, 0);
1797 jl(LabelNeg); // newx < 0, read (partially or fully) from before the leftmost pixel
1798 }
1799 */
1800
1801 // It's safe to read the whole pixel group
1802 int offset;
1803 ✗ if (iter.op == opLoadRelSrc8)
1804 ✗ offset = iter.dx;
1805 ✗ else if (iter.op == opLoadRelSrc16)
1806 ✗ offset = iter.dx * sizeof(uint16_t);
1807 ✗ else if (iter.op == opLoadRelSrcF32)
1808 ✗ offset = iter.dx * sizeof(float);
1809
1810 ✗ if (iter.op == opLoadRelSrc8) {
1811 ✗ movq(r1, mmword_ptr[a + offset]); // 8 pixels, 8 bytes
1812 ✗ punpcklbw(r1, zero);
1813 ✗ movdqa(r2, r1);
1814 ✗ punpcklwd(r1, zero);
1815 ✗ punpckhwd(r2, zero);
1816 ✗ cvtdq2ps(r1, r1);
1817 ✗ cvtdq2ps(r2, r2);
1818 }
1819 ✗ else if (iter.op == opLoadRelSrc16) {
1820 ✗ if (iter.dx % 8 == 0)
1821 ✗ movdqa(r1, xmmword_ptr[a + offset]); // 8 pixels 16 byte boundary, aligned
1822 else
1823 ✗ movdqu(r1, xmmword_ptr[a + offset]);
1824 ✗ movdqa(r2, r1);
1825 ✗ punpcklwd(r1, zero);
1826 ✗ punpckhwd(r2, zero);
1827 ✗ cvtdq2ps(r1, r1);
1828 ✗ cvtdq2ps(r2, r2);
1829 }
1830 ✗ else if (iter.op == opLoadRelSrcF32) {
1831 ✗ if (iter.dx % 4 == 0) {
1832 ✗ movdqa(r1, xmmword_ptr[a + offset]); // // 4 pixels 16 byte boundary, aligned
1833 ✗ movdqa(r2, xmmword_ptr[a + offset + 16]);
1834 }
1835 else {
1836 ✗ movdqu(r1, xmmword_ptr[a + offset]); // unaligned
1837 ✗ movdqu(r2, xmmword_ptr[a + offset + 16]);
1838 }
1839 }
1840 ✗ if (iter.dx != 0) {
1841 ✗ jmp(LabelEnd); // Optim: generate jump only when over/negative branches exist
1842 }
1843
1844 ✗ if (iter.dx != 0) {
1845 ✗ L(LabelOver);
1846
1847 // x dx newx
1848 // 8 1 8+1=9
1849 // planewidth == 16
1850 // 0 1 2 3 4 5 6 7 8 9 A B C D E F g h i j
1851 // P Q R S T U V x need this
1852 // P Q R S T U V V duplicated the last valid pixel
1853 // 0 1 2 3 4 5 6 7 we can read this
1854 // 1 2 3 4 5 6 7 - Shift right by dx to lose low bytes
1855 // 1 2 3 4 5 6 7 7 have to make this one from it. Only the first planewidth-newx (7) bytes valid
1856 // x dx newx
1857 // 8 3 8+3=11
1858 // planewidth == 15
1859 // 0 1 2 3 4 5 6 7 8 9 A B C D E f g h i j
1860 // P Q R S T x x x need this
1861 // P Q R S S S S S duplicated the last valid pixel
1862 // 0 1 2 3 4 5 6 7 we can read this
1863 // 3 4 5 6 7 - - - Shift right by dx to lose low bytes
1864 // 3 4 5 6 6 6 6 6 have to make this one from it. Only the first planewidth-newx (4) bytes valid
1865 // planewidth == 14
1866 // 0 1 2 3 4 5 6 7 8 9 A B C D e f g h i j
1867 // P Q R x x x x x need this
1868 // P Q R R R R R R duplicated the last valid pixel
1869 // 0 1 2 3 4 5 6 7 we can read this
1870 // 3 4 5 6 7 - - - Shift right by dx to lose low bytes
1871 // 3 4 5 5 5 5 5 5 have to make this one from it. Only the first planewidth-newx (3) bytes valid
1872 // special case: full read from beyond line
1873 // planewidth == 14 dx=6 newx = 8+6=14, newx>=planewidth => newx = 13 => planewidth-newx = 1
1874 // 0 1 2 3 4 5 6 7 8 9 A B C D e f g h i j k l
1875 // ? ? ? ? ? ? ? ? need this, but overread
1876 // ? ? ? ? ? ? ? ? duplicated the last valid pixel
1877 // 0 1 2 3 4 5 6 7 we can read this
1878 // 5 6 7 - - - - - Shift right by not dx but (planewidth-1)&7, it's 5 in this example, to lose low bytes
1879 // 5 5 5 5 5 5 5 5 case(1): duplicate very first
1880 // planewidth == 14 dx=7 newx = 0+7=7, newx>=planewidth-7, then not newx>=planewidth => newx = 13 => planewidth-newx = 1
1881 // 0 1 2 3 4 5 6 7 8 9 A B C D e f g h i j k l
1882 // P Q R S T U V w need this
1883 // P Q R S T U V V duplicated the last valid pixel
1884 // 0 1 2 3 4 5 6 7 we can read this
1885 // 5 6 7 - - - - - Shift right by not dx but (planewidth-1)&7, it's 5 in this example, to lose low bytes
1886 // 5 5 5 5 5 5 5 5 case(1): duplicate very first
1887 // planewidth == 14 dx=7 newx = 8+7=15, newx>=planewidth => newx = 13 => planewidth-newx = 1
1888 // ? ? ? ? ? ? ? ? need this, but overread
1889 // 0 1 2 3 4 5 6 7 we can read this
1890 // 5 - - - - - - - Shift right to have the last pixel
1891 // 5 5 5 5 5 5 5 5 case(1): duplicate very first
1892 // planewidth == 13
1893 // 0 1 2 3 4 5 6 7 8 9 A B C d e f g h i j
1894 // P Q x x x x x x need this
1895 // P Q Q Q Q Q Q Q duplicated the last valid pixel
1896 // 0 1 2 3 4 5 6 7 we can read this
1897 // 3 4 5 6 7 - - - Shift right by dx to lose low bytes
1898 // 3 4 4 4 4 4 4 4 have to make this one from it. Only the first planewidth-newx (2) bytes valid
1899 // planewidth == 12
1900 // 0 1 2 3 4 5 6 7 8 9 A B c d e f g h i j
1901 // P x x x x x x x need this
1902 // P P P P P P P P duplicated the last valid pixel
1903 // 0 1 2 3 4 5 6 7 we can read this
1904 // 3 4 5 6 7 - - - Shift right by dx to lose low bytes
1905 // 3 3 3 3 3 3 3 3 have to make this one from it. Only the first planewidth-newx (1) bytes valid
1906
1907 // duplicate highest, make a shuffle table by planewidth-newx (1..7)
1908 // planewidth - newx newx-pw newx-pw+7 shuffle
1909 // newx-(pw-7)
1910 // 1 -1 6 0->0 0->1 0->2 0->3 0->4 0->5 0->6 0->7 elSuffleForRight6 (lowest to everywhere)
1911 // 2 -2 5 0->0 1->1 1->2 1->3 1->4 1->5 1->6 1->7 elSuffleForRight5 (lowest two remains then second duplicates)
1912 // 3 -3 4 0->0 1->1 2->2 2->3 2->4 2->5 2->6 2->7 elSuffleForRight4 (lowest three remains then third duplicates)
1913 // 4 -4 3 0->0 1->1 2->2 3->3 3->4 3->5 3->6 3->7 elSuffleForRight3
1914 // 5 -5 2 0->0 1->1 2->2 3->3 4->4 4->5 4->6 4->7 elSuffleForRight2
1915 // 6 -6 1 0->0 1->1 2->2 3->3 4->4 5->5 5->6 5->7 elSuffleForRight1
1916 // 7 -7 0 0->0 1->1 2->2 3->3 4->4 5->5 6->6 6->7 elSuffleForRight0 (lowest seven remains then seventh duplicates)
1917 // in extreme case (read all beyond last pixel): newx >= planewidth: ==> case of elSuffleForRight6
1918
1919 // shuffleTable = elShuffleForRight0 + 16*(newx-(planewidth-7))
1920 ✗ std::string PartialOverread = "PartialOverread" + getLabelCount();
1921 ✗ std::string NoFullOverReadFromNewX = "NoFullOverReadFromNewX" + getLabelCount();
1922 ✗ std::string labelDoOver = "DoOver" + getLabelCount();
1923
1924 ✗ if (iter.dx > 0) { // FullOverRead possible only when dx>0
1925 ✗ cmp(newx, planewidth);
1926 ✗ jl(PartialOverread); // if newx < planewidth ->
1927
1928 // case: FullOver
1929 // even the first pixel to read is beyond the end of line
1930 // We have to clone the rightmost pixel from (planewidth-1)
1931 ✗ if (iter.op == opLoadRelSrc8) {
1932 ✗ sub(a, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1933 ✗ add(a, planewidth - 1);
1934 // reuse newx
1935 ✗ movzx(newx, byte_ptr[a]);
1936 ✗ movd(r1, newx);
1937 ✗ punpcklbw(r1, zero); // words
1938 ✗ pshufb(r1, CPTR(elShuffleForRight6)); // duplicate last word to all
1939
1940 ✗ movdqa(r2, r1);
1941 ✗ punpcklwd(r1, zero);
1942 ✗ punpckhwd(r2, zero);
1943 ✗ cvtdq2ps(r1, r1);
1944 ✗ cvtdq2ps(r2, r2);
1945 }
1946 ✗ else if (iter.op == opLoadRelSrc16) {
1947 ✗ Reg tmp;
1948 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1949 ✗ shl(tmp, 1); // for 16 bit 2*xcounter
1950 ✗ sub(a, tmp);
1951 ✗ add(a, (planewidth - 1) * 2);
1952 // reuse newx
1953 ✗ movzx(newx, word_ptr[a]);
1954 ✗ movd(r1, newx);
1955 ✗ pshufb(r1, CPTR(elShuffleForRight6)); // duplicate last word to all
1956
1957 ✗ movdqa(r2, r1);
1958 ✗ punpcklwd(r1, zero);
1959 ✗ punpckhwd(r2, zero);
1960 ✗ cvtdq2ps(r1, r1);
1961 ✗ cvtdq2ps(r2, r2);
1962 }
1963 ✗ else if (iter.op == opLoadRelSrcF32) {
1964 ✗ Reg tmp;
1965 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
1966 ✗ shl(tmp, 2); // for 32 bit 4*xcounter
1967 ✗ sub(a, tmp);
1968 ✗ add(a, (planewidth - 1) * 4);
1969 ✗ movd(r1, dword_ptr[a]);
1970 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6));
1971 ✗ movdqa(r2, r1);
1972 }
1973 ✗ jmp(LabelEnd);
1974 } // full OverRead path, needed when iter.dx>0
1975
1976 // case: Partial overread
1977 // read the block, then clone the last valid pixel from position (planewidth-1)
1978 // problem: newx is not aligned
1979 ✗ L(PartialOverread);
1980 // planewidth == 14 dx=7 newx = 0+7=7, newx>=planewidth-7, then not newx>=planewidth => newx = 13 => planewidth-newx = 1
1981 // sample 1: newx is in different segment than planewidth-1
1982 // [xcounter]
1983 // [a] [newx] [pw-1]
1984 // V V V
1985 // 0 1 2 3 4 5 6 7 8 9 A B C D e f g h i j k l
1986 // P Q R S T U V w need this
1987 // 0 1 2 3 4 5 6 7 we can read this
1988 // P Q R S T U V w last pixel is beyond
1989 // P Q R S T U V V need this
1990 // sample 2: newx is in the same segment than planewidth-1
1991 // planewidth == 13 dx=3
1992 // [xcounter] [newx]
1993 // [a][newx][pw-1]
1994 // V V V
1995 // 0 1 2 3 4 5 6 7 8 9 A B C d e f g h i j
1996 // P Q x x x x x x need this
1997 // P Q Q Q Q Q Q Q duplicated the last valid pixel
1998 // 0 1 2 3 4 5 6 7 we can read this
1999 // when newx and (planewidth-1) are in different segments then we read from newx
2000 ✗ Reg tmp;
2001 ✗ mov(tmp, newx);
2002 ✗ and_(tmp, ~(pixels_per_cycle - 1));
2003 ✗ cmp(tmp, (planewidth & ~(pixels_per_cycle - 1)));
2004 ✗ jle(NoFullOverReadFromNewX); // jump if (newx and ~0x07) < (planewidth & ~0x07) (in another segment)
2005
2006 // read from current (last) pointer,
2007 ✗ if (iter.op == opLoadRelSrc8 || iter.op == opLoadRelSrc16) {
2008 ✗ if (iter.op == opLoadRelSrc8) {
2009 ✗ movq(r1, mmword_ptr[a]); // 8 pixels, 8 bytes
2010 ✗ punpcklbw(r1, zero); // words
2011 }
2012 else { // opLoadRel16
2013 ✗ movdqa(r1, xmmword_ptr[a]); // 8 pixels, 16 bytes, here still aligned
2014 }
2015 ✗ psrldq(r1, ((planewidth - 1) & (pixels_per_cycle - 1)) * sizeof(uint16_t)); // Shift right by (planewidth - 1) & 7 to lose low words
2016 ✗ sub(newx, planewidth - (pixels_per_cycle - 1)); // find out shuffle pointer -1, ... -7 -> 6 ... 0
2017 ✗ shl(newx, 4); // *16 for shuffle table
2018 // LabelDoOver copied here
2019 // reuse a : Reg shuffleTable;
2020 ✗ lea(a/*shuffleTable*/, CPTR(elShuffleForRight0)); // ptr for word shuffle
2021 //add(a/*shuffleTable*/, newx);
2022 ✗ pshufb(r1, xmmword_ptr[a/*shuffleTable*/ + newx]);
2023
2024 ✗ movdqa(r2, r1);
2025 ✗ punpcklwd(r1, zero);
2026 ✗ punpckhwd(r2, zero);
2027 ✗ cvtdq2ps(r1, r1);
2028 ✗ cvtdq2ps(r2, r2);
2029 ✗ jmp(LabelEnd);
2030 }
2031 ✗ else if (iter.op == opLoadRelSrcF32) {
2032 // omg it's complicated
2033
2034 // palignr memo
2035 // temp1[255:0] ((DEST[127:0] << 128) OR SRC[127:0]) >> (imm8*8);
2036 // DEST[127:0] temp1[127:0]
2037
2038 ✗ int bytes_to_shift = ((planewidth - 1) & (pixels_per_cycle - 1)) * sizeof(float);
2039 ✗ if (bytes_to_shift < 16) {
2040 // src dst
2041 // r2 r1
2042 // 15 14 13.... 0 15 14 13 ... 0
2043 // 15 14 13 1 0 15 14.... 1 palignr(dst, src, 1)
2044 // 15 14 13 ..... 15 palignr(dst, src, 15)
2045 ✗ movdqa(r1, xmmword_ptr[a]); // 4 pixels, 16 bytes, here still aligned
2046 ✗ movdqa(r2, xmmword_ptr[a + 16]); // 4 pixels, 16 bytes
2047 ✗ if (bytes_to_shift > 0) {
2048 ✗ palignr(r1, r2, bytes_to_shift); // shift right dualreg. r1 is ready.
2049 ✗ psrldq(r2, bytes_to_shift); // Shift right upper part
2050 ✗ switch (bytes_to_shift) { // 4, 8, 12
2051 ✗ case 4:
2052 ✗ pshufd(r2, r2, (0 << 0) | (1 << 2) | (2 << 4) | (2 << 6)); // elShuffleForRight0_32
2053 ✗ break;
2054 ✗ case 8:
2055 ✗ pshufd(r2, r2, (0 << 0) | (1 << 2) | (1 << 4) | (1 << 6));
2056 ✗ break;
2057 ✗ case 12:
2058 ✗ pshufd(r2, r2, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6)); // elShuffleForRight2_32
2059 ✗ break;
2060 }
2061 }
2062 }
2063 ✗ else if (bytes_to_shift == 16) {
2064 // src dst
2065 // r2 r1
2066 // 15 14 13.... 0 15 14 13 ... 0 --> 16 bytes: r1 = r2
2067 ✗ movdqa(r1, xmmword_ptr[a + 16]); // 4 pixels, 16 bytes, no need [a + 0], here still aligned
2068 ✗ pshufd(r2, r1, (3 << 0) | (3 << 2) | (3 << 4) | (3 << 6)); // fill r2 with highest dword of r1
2069 }
2070 else {
2071 // bytes to shift > 16 (20, 24, 28), ignore lower 4 pixels, move and shift and spread from upper 4 pixels
2072 ✗ movdqa(r1, xmmword_ptr[a + 16]); // 4 pixels, 16 bytes, no need [a + 0], here still aligned
2073 ✗ psrldq(r1, bytes_to_shift - 16); // Shift right upper part
2074 ✗ switch (bytes_to_shift) { // 0, 4, 8, 12
2075 ✗ case 20:
2076 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (2 << 4) | (2 << 6)); // elShuffleForRight0_32
2077 ✗ break;
2078 ✗ case 24:
2079 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (1 << 4) | (1 << 6));
2080 ✗ break;
2081 ✗ case 28:
2082 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6)); // elShuffleForRight2_32
2083 ✗ break;
2084 }
2085 ✗ pshufd(r2, r1, (3 << 0) | (3 << 2) | (3 << 4) | (3 << 6)); // fill r2 with highest dword of r1
2086 }
2087 ✗ jmp(LabelEnd);
2088 }
2089
2090 ✗ L(NoFullOverReadFromNewX);
2091 // read from newx
2092 //a = a - 1,2,4 * xcounter + 1,2,4 * newx;
2093 ✗ sub(newx, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
2094 ✗ if (iter.op == opLoadRelSrc8 || iter.op == opLoadRelSrc16) {
2095 ✗ if (iter.op == opLoadRelSrc8)
2096 {
2097 ✗ add(a, newx); // new position
2098 ✗ movq(r1, mmword_ptr[a]); // 8 pixels, 8 bytes
2099 ✗ punpcklbw(r1, zero); // words
2100 }
2101 else {
2102 ✗ shl(newx, 1); //a = a - 2 * xcounter + 2 * newx;
2103 ✗ add(a, newx);
2104 ✗ movdqu(r1, xmmword_ptr[a]); // 8 pixels, 16 bytes
2105 }
2106 // no shift here, just duplicate appropriate pixel into the high ones
2107 ✗ int what = ((planewidth - iter.dx - 1) & (pixels_per_cycle - 1));
2108 ✗ switch (what) {
2109 ✗ case 0:
2110 ✗ pshufb(r1, CPTR(elShuffleForRight6));
2111 ✗ break;
2112 ✗ case 1:
2113 ✗ pshufb(r1, CPTR(elShuffleForRight5));
2114 ✗ break;
2115 ✗ case 2:
2116 ✗ pshufb(r1, CPTR(elShuffleForRight4));
2117 ✗ break;
2118 ✗ case 3:
2119 ✗ pshufb(r1, CPTR(elShuffleForRight3));
2120 ✗ break;
2121 ✗ case 4:
2122 ✗ pshufb(r1, CPTR(elShuffleForRight2));
2123 ✗ break;
2124 ✗ case 5:
2125 ✗ pshufb(r1, CPTR(elShuffleForRight1));
2126 ✗ break;
2127 ✗ case 6:
2128 ✗ pshufb(r1, CPTR(elShuffleForRight0));
2129 ✗ break;
2130 }
2131 ✗ movdqa(r2, r1);
2132 ✗ punpcklwd(r1, zero);
2133 ✗ punpckhwd(r2, zero);
2134 ✗ cvtdq2ps(r1, r1);
2135 ✗ cvtdq2ps(r2, r2);
2136 //jmp(LabelEnd);
2137 ✗ }
2138 ✗ else if (iter.op == opLoadRelSrcF32) {
2139 ✗ shl(newx, 2); // float: *4
2140 ✗ add(a, newx);
2141
2142 ✗ int what = ((planewidth - iter.dx - 1) & (pixels_per_cycle - 1));
2143 // what
2144 // 0 ShuffleForRight2_32 keep r1 dword #0 , spread it to 1..3, then spread r1.3 to r2 (ShuffleForRight2_32(r2,r1)
2145 // 1 ShuffleForRight1_32 keep r1 dword #0..1, spread it to 2..3, then spread r1.3 to r2
2146 // 2 ShuffleForRight0_32 keep r1 dword #0..2, spread it to 3 , then spread r1.3 to r2
2147 // 3 keep r1 dword #0..3, , then spread r1.3 to r2
2148 // 4 keep r1 dword #0..3, , keep r2 dword #0 , spread it to 1..3
2149 // 5 keep r1 dword #0..3, , keep r2 dword #0..1, spread it to 2..3
2150 // 6 keep r1 dword #0..3, , keep r2 dword #0..2, spread it to 3
2151
2152 // no real shift here, just duplicate appropriate pixel into the high ones. But we have two registers
2153 ✗ if (what <= 3) {
2154 ✗ movdqu(r1, xmmword_ptr[a]); // 4 pixels, 16 bytes, no need for upper 4 pixels
2155 ✗ switch (what) {
2156 ✗ case 0:
2157 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6)); // elShuffleForRight2_32 // fill 3 upper dwords of r1 from r1.0
2158 ✗ break;
2159 ✗ case 1:
2160 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (1 << 4) | (1 << 6)); // fill 2 upper dwords of r1 from r1.1
2161 ✗ break;
2162 ✗ case 2:
2163 ✗ pshufd(r1, r1, (0 << 0) | (1 << 2) | (2 << 4) | (2 << 6)); // elShuffleForRight0_32 // fill 1 upper dwords of r1 from r1.2
2164 ✗ break;
2165 }
2166 ✗ pshufd(r2, r1, (3 << 0) | (3 << 2) | (3 << 4) | (3 << 6)); // fill all dwords of r2 from r1.3
2167 }
2168 else {
2169 ✗ movdqu(r1, xmmword_ptr[a]); // 4 pixels, 16 bytes, low 4 pixels keep them as is
2170 ✗ movdqu(r2, xmmword_ptr[a + 16]); // 4 pixels, 16 bytes
2171 ✗ switch (what) {
2172 ✗ case 4:
2173 ✗ pshufd(r2, r2, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6)); // elShuffleForRight2_32 // fill 3 upper dwords of r1 from r2.0
2174 ✗ break;
2175 ✗ case 5:
2176 ✗ pshufd(r2, r2, (0 << 0) | (1 << 2) | (1 << 4) | (1 << 6)); // fill 2 upper dwords of r1 from r2.1
2177 ✗ break;
2178 ✗ case 6:
2179 ✗ pshufd(r2, r2, (0 << 0) | (1 << 2) | (2 << 4) | (2 << 6)); // elShuffleForRight0_32 // fill 1 upper dwords of r1 from r2.2
2180 ✗ break;
2181 }
2182 }
2183 // continues on labelEnd
2184 }
2185 ✗ if (iter.dx < 0)
2186 ✗ jmp(LabelEnd);
2187 ✗ } // over: iter.dx != 0
2188 ✗ if (iter.dx < 0) {
2189 ✗ L(LabelNeg);
2190 // When reading from negative x coordinates we read exactly from 0th, then shift and duplicate
2191 // For extreme minus offsets we duplicate 0th (leftmost) pixel to each position
2192 // example: dx = -1
2193 // -1 0 1 2 3 4 5 6 7
2194 // A A B C D E F G we need this
2195 // A B C D E F G H read [0]
2196 // 0 A B C D E F G H shift
2197 // A A B C D E F G H duplicate by shuffle
2198 ✗ if (iter.op == opLoadRelSrc8 || iter.op == opLoadRelSrc16) {
2199 ✗ if (iter.op == opLoadRelSrc8) {
2200 ✗ sub(a, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]); // go back to the beginning
2201 ✗ movq(r1, mmword_ptr[a]); // 8 pixels, 8 bytes
2202 ✗ punpcklbw(r1, zero); // bytes to words
2203 }
2204 ✗ else if (iter.op == opLoadRelSrc16) {
2205 // go back to the beginning, in 16 bit, *2
2206 ✗ Reg tmp;
2207 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
2208 ✗ shl(tmp, 1); // for 16 bit 2*xcounter
2209 ✗ sub(a, tmp);
2210 ✗ movdqa(r1, xmmword_ptr[a]); // 8 pixels, 16 bytes
2211 }
2212
2213 ✗ std::string PartialReadBefore = "PartialReadBefore" + getLabelCount();
2214 ✗ std::string Finalize = "Finalize" + getLabelCount();
2215 ✗ cmp(newx, -pixels_per_cycle); // pixels_per_cycle words
2216 ✗ jg(PartialReadBefore);
2217 // FullReadBefore: newx <= -pixels_per_cycle, clone 0th (leftmost) pixel to all
2218 ✗ pshufb(r1, CPTR(elShuffleForRight6)); // lowest word to all
2219 ✗ jmp(Finalize);
2220 ✗ L(PartialReadBefore);
2221 // -pixels_per_cycle < newx < 0
2222 ✗ int toShift = min(pixels_per_cycle - 1, (-iter.dx) & (pixels_per_cycle - 1));
2223 ✗ pslldq(r1, toShift * 2); // shift in word domain
2224 ✗ switch (toShift) {
2225 ✗ case 1: pshufb(r1, CPTR(elShuffleForLeft0)); break;
2226 ✗ case 2: pshufb(r1, CPTR(elShuffleForLeft1)); break;
2227 ✗ case 3: pshufb(r1, CPTR(elShuffleForLeft2)); break;
2228 ✗ case 4: pshufb(r1, CPTR(elShuffleForLeft3)); break;
2229 ✗ case 5: pshufb(r1, CPTR(elShuffleForLeft4)); break;
2230 ✗ case 6: pshufb(r1, CPTR(elShuffleForLeft5)); break;
2231 ✗ case 7: pshufb(r1, CPTR(elShuffleForLeft6)); break;
2232 }
2233 ✗ L(Finalize);
2234
2235 ✗ movdqa(r2, r1);
2236 ✗ punpcklwd(r1, zero);
2237 ✗ punpckhwd(r2, zero);
2238 ✗ cvtdq2ps(r1, r1);
2239 ✗ cvtdq2ps(r2, r2);
2240 ✗ }
2241 ✗ else if (iter.op == opLoadRelSrcF32) {
2242 // negative
2243 // go back to the beginning, in 16 bit, *2
2244 ✗ Reg tmp;
2245 ✗ mov(tmp, ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
2246 ✗ shl(tmp, 2); // go back to the beginning, in 32 bit, *4
2247 ✗ sub(a, tmp);
2248
2249 ✗ std::string PartialReadBefore = "PartialReadBefore" + getLabelCount();
2250
2251 ✗ cmp(newx, -pixels_per_cycle);
2252 ✗ jg(PartialReadBefore);
2253 // FullReadBefore: newx <= -pixels_per_cycle, clone 0th (leftmost) pixel to all
2254 ✗ movdqa(r1, xmmword_ptr[a]);
2255 ✗ pshufd(r1, r1, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6));
2256 ✗ movdqa(r2, r1);
2257 ✗ jmp(LabelEnd);
2258
2259 ✗ L(PartialReadBefore);
2260 // -pixels_per_cycle < newx < 0
2261 ✗ int bytes_to_shift = sizeof(float) * min(pixels_per_cycle - 1, (-iter.dx) & (pixels_per_cycle - 1));
2262 // shift bytes
2263 // 4 r2r1 << 4 shuffle r1.1 to r1.0-0
2264 // 8 r2r1 << 8 shuffle r1.2 to r1.0-1
2265 // 12 r2r1 << 12 shuffle r1.3 to r1.0-2
2266 // 16 r2r1 << 16 -> r2 = r1, , shuffle r2.0 to all r1
2267 // 20 r2r1 << 20 -> r2 = r1, r2 << (20-4), shuffle r2.1 to r2.0-0, shuffle r2.0 to all r1
2268 // 24 r2r1 << 24 -> r2 = r1, r2 << (24-4), shuffle r2.2 to r2.0-1, shuffle r2.0 to all r1
2269 // 28 r2r1 << 28 -> r2 = r1, r2 << (28-4), shuffle r2.3 to r2.0-2, shuffle r2.0 to all r1
2270 ✗ if (bytes_to_shift < 16) {
2271 ✗ movdqa(r1, xmmword_ptr[a]); // 4 pixels, 16 bytes
2272 ✗ movdqa(r2, xmmword_ptr[a + 16]); // 4 pixels, 16 bytes
2273 // 4 floats
2274 // r2 r1
2275 // H3 H2 H1 H0 L3 L2 L1 L0 << 1*4 byte
2276 // H2 H1 H0 L3 L2 L1 L0 00
2277 // H2 H1 H0 00 or
2278 // 00 00 00 L3 L2 L1 L0 00
2279 ✗ psrldq(r1, 16 - bytes_to_shift);
2280 ✗ pslldq(r2, bytes_to_shift);
2281 ✗ por(r2, r1);
2282 ✗ movdqa(r1, xmmword_ptr[a]); // load again
2283 ✗ pslldq(r1, bytes_to_shift); // todo: shift + shuffle = single shuffle
2284
2285 ✗ switch (bytes_to_shift) { // 4, 8, 12
2286 ✗ case 4:
2287 ✗ pshufd(r1, r1, (1 << 0) | (1 << 2) | (2 << 4) | (3 << 6)); // elShuffleForLeft0_32 // shuffle r1.1 to r1.0-0
2288 ✗ break;
2289 ✗ case 8:
2290 ✗ pshufd(r1, r1, (2 << 0) | (2 << 2) | (2 << 4) | (3 << 6)); // elShuffleForLeft1_32 // shuffle r1.2 to r1.0-1
2291 ✗ break;
2292 ✗ case 12:
2293 ✗ pshufd(r1, r1, (3 << 0) | (3 << 2) | (3 << 4) | (3 << 6)); // elShuffleForLeft2_32 // shuffle r1.3 to r1.0-2
2294 ✗ break;
2295 }
2296 }
2297 else {
2298 // toShift >= 16
2299 //movdqa(r1, xmmword_ptr[a]); // no need for 15..31
2300 ✗ movdqa(r2, xmmword_ptr[a]); // 4 pixels, 16 bytes
2301 ✗ if (bytes_to_shift > 16)
2302 ✗ pslldq(r2, bytes_to_shift - 16);
2303
2304 ✗ switch (bytes_to_shift) { // 20, 24, 28
2305 ✗ case 20:
2306 ✗ pshufd(r2, r2, (1 << 0) | (1 << 2) | (2 << 4) | (3 << 6)); // elShuffleForLeft0_32 // shuffle r2.1 to r2.0-0
2307 ✗ break;
2308 ✗ case 24:
2309 ✗ pshufd(r2, r2, (2 << 0) | (2 << 2) | (2 << 4) | (3 << 6)); // elShuffleForLeft1_32 // shuffle r2.2 to r2.0-1
2310 ✗ break;
2311 ✗ case 28:
2312 ✗ pshufd(r2, r2, (3 << 0) | (3 << 2) | (3 << 4) | (3 << 6)); // elShuffleForLeft2_32 // shuffle r2.3 to r2.0-2
2313 ✗ break;
2314 }
2315 ✗ pshufd(r1, r2, (0 << 0) | (0 << 2) | (0 << 4) | (0 << 6)); // shuffle r2.0 to all r1
2316 }
2317 ✗ }
2318 }
2319 ✗ L(LabelEnd);
2320 ✗ stack.push_back(std::make_pair(r1, r2));
2321 ✗ }
2322 ✗ } // oploadRel8/16/32
2323
5/10
✗ Branch 396 → 397 not taken.
✗ Branch 396 → 410 not taken.
✓ Branch 4954 → 4955 taken 3 times.
✓ Branch 4954 → 4973 taken 22 times.
✓ Branch 13070 → 13071 taken 3 times.
✓ Branch 13070 → 13111 taken 7 times.
✗ Branch 21054 → 21055 not taken.
✗ Branch 21054 → 21068 not taken.
✗ Branch 25382 → 25383 not taken.
✓ Branch 25382 → 25418 taken 12 times.
 47 else if (iter.op == opLoadSrc8) {
2324 if (processSingle) {
2325 ✗ XmmReg r1;
2326 ✗ Reg a;
2327 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2328 ✗ movd(r1, dword_ptr[a]); // 4 pixels, 4 bytes
2329 ✗ punpcklbw(r1, zero);
2330 ✗ punpcklwd(r1, zero);
2331 ✗ cvtdq2ps(r1, r1);
2332 ✗ if (maskIt)
2333 ✗ doMask(r1, constptr, planewidth);
2334 ✗ stack1.push_back(r1);
2335 }
2336 else {
2337
4/8
✓ Branch 4955 → 4956 taken 3 times.
✗ Branch 4955 → 10955 not taken.
✓ Branch 4956 → 4957 taken 3 times.
✗ Branch 4956 → 10955 not taken.
✓ Branch 13071 → 13072 taken 3 times.
✗ Branch 13071 → 19200 not taken.
✓ Branch 13072 → 13073 taken 3 times.
✗ Branch 13072 → 19200 not taken.
 6 XmmReg r1, r2;
2338
2/4
✓ Branch 4957 → 4958 taken 3 times.
✗ Branch 4957 → 10955 not taken.
✓ Branch 13073 → 13074 taken 3 times.
✗ Branch 13073 → 19200 not taken.
 6 Reg a;
2339
4/8
✓ Branch 4959 → 4960 taken 3 times.
✗ Branch 4959 → 10950 not taken.
✓ Branch 4960 → 4961 taken 3 times.
✗ Branch 4960 → 10950 not taken.
✓ Branch 13075 → 13076 taken 3 times.
✗ Branch 13075 → 19195 not taken.
✓ Branch 13076 → 13077 taken 3 times.
✗ Branch 13076 → 19195 not taken.
 6 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2340
4/8
✓ Branch 4962 → 4963 taken 3 times.
✗ Branch 4962 → 10952 not taken.
✓ Branch 4963 → 4964 taken 3 times.
✗ Branch 4963 → 10952 not taken.
✓ Branch 13078 → 13079 taken 3 times.
✗ Branch 13078 → 19197 not taken.
✓ Branch 13079 → 13080 taken 3 times.
✗ Branch 13079 → 19197 not taken.
 6 movq(r1, mmword_ptr[a]);
2341
2/4
✓ Branch 4964 → 4965 taken 3 times.
✗ Branch 4964 → 10955 not taken.
✓ Branch 13080 → 13081 taken 3 times.
✗ Branch 13080 → 19200 not taken.
 6 punpcklbw(r1, zero);
2342
2/4
✓ Branch 4965 → 4966 taken 3 times.
✗ Branch 4965 → 10955 not taken.
✓ Branch 13081 → 13082 taken 3 times.
✗ Branch 13081 → 19200 not taken.
 6 movdqa(r2, r1);
2343
2/4
✓ Branch 4966 → 4967 taken 3 times.
✗ Branch 4966 → 10955 not taken.
✓ Branch 13082 → 13083 taken 3 times.
✗ Branch 13082 → 19200 not taken.
 6 punpcklwd(r1, zero);
2344
2/4
✓ Branch 4967 → 4968 taken 3 times.
✗ Branch 4967 → 10955 not taken.
✓ Branch 13083 → 13084 taken 3 times.
✗ Branch 13083 → 19200 not taken.
 6 punpckhwd(r2, zero);
2345
2/4
✓ Branch 4968 → 4969 taken 3 times.
✗ Branch 4968 → 10955 not taken.
✓ Branch 13084 → 13085 taken 3 times.
✗ Branch 13084 → 19200 not taken.
 6 cvtdq2ps(r1, r1);
2346
2/4
✓ Branch 4969 → 4970 taken 3 times.
✗ Branch 4969 → 10955 not taken.
✓ Branch 13085 → 13086 taken 3 times.
✗ Branch 13085 → 19200 not taken.
 6 cvtdq2ps(r2, r2);
2347
1/2
✓ Branch 13086 → 13087 taken 3 times.
✗ Branch 13086 → 13108 not taken.
 3 if (maskIt)
2348
1/4
✗ Branch 13087 → 13088 not taken.
✗ Branch 13087 → 13092 not taken.
✓ Branch 13087 → 13096 taken 3 times.
✗ Branch 13087 → 13100 not taken.
 3 doMask(r2, constptr, planewidth);
2349
4/8
✓ Branch 4970 → 4971 taken 3 times.
✗ Branch 4970 → 10954 not taken.
✓ Branch 4971 → 4972 taken 3 times.
✗ Branch 4971 → 10954 not taken.
✓ Branch 13108 → 13109 taken 3 times.
✗ Branch 13108 → 19199 not taken.
✓ Branch 13109 → 13110 taken 3 times.
✗ Branch 13109 → 19199 not taken.
 6 stack.push_back(std::make_pair(r1, r2));
2350 }
2351 }
2352
5/10
✗ Branch 410 → 411 not taken.
✗ Branch 410 → 423 not taken.
✓ Branch 4973 → 4974 taken 3 times.
✓ Branch 4973 → 4991 taken 19 times.
✗ Branch 13111 → 13112 not taken.
✓ Branch 13111 → 13151 taken 7 times.
✗ Branch 21068 → 21069 not taken.
✗ Branch 21068 → 21081 not taken.
✓ Branch 25418 → 25419 taken 3 times.
✓ Branch 25418 → 25453 taken 9 times.
 41 else if (iter.op == opLoadSrc16) {
2353 if (processSingle) {
2354
1/6
✗ Branch 411 → 412 not taken.
✗ Branch 411 → 3585 not taken.
✗ Branch 21069 → 21070 not taken.
✗ Branch 21069 → 24243 not taken.
✓ Branch 25419 → 25420 taken 3 times.
✗ Branch 25419 → 28699 not taken.
 3 XmmReg r1;
2355
1/6
✗ Branch 412 → 413 not taken.
✗ Branch 412 → 3585 not taken.
✗ Branch 21070 → 21071 not taken.
✗ Branch 21070 → 24243 not taken.
✓ Branch 25420 → 25421 taken 3 times.
✗ Branch 25420 → 28699 not taken.
 3 Reg a;
2356
2/12
✗ Branch 414 → 415 not taken.
✗ Branch 414 → 3581 not taken.
✗ Branch 415 → 416 not taken.
✗ Branch 415 → 3581 not taken.
✗ Branch 21072 → 21073 not taken.
✗ Branch 21072 → 24239 not taken.
✗ Branch 21073 → 21074 not taken.
✗ Branch 21073 → 24239 not taken.
✓ Branch 25422 → 25423 taken 3 times.
✗ Branch 25422 → 28695 not taken.
✓ Branch 25423 → 25424 taken 3 times.
✗ Branch 25423 → 28695 not taken.
 3 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2357
2/12
✗ Branch 417 → 418 not taken.
✗ Branch 417 → 3583 not taken.
✗ Branch 418 → 419 not taken.
✗ Branch 418 → 3583 not taken.
✗ Branch 21075 → 21076 not taken.
✗ Branch 21075 → 24241 not taken.
✗ Branch 21076 → 21077 not taken.
✗ Branch 21076 → 24241 not taken.
✓ Branch 25425 → 25426 taken 3 times.
✗ Branch 25425 → 28697 not taken.
✓ Branch 25426 → 25427 taken 3 times.
✗ Branch 25426 → 28697 not taken.
 3 movq(r1, mmword_ptr[a]); // 4 pixels, 8 bytes
2358
1/6
✗ Branch 419 → 420 not taken.
✗ Branch 419 → 3585 not taken.
✗ Branch 21077 → 21078 not taken.
✗ Branch 21077 → 24243 not taken.
✓ Branch 25427 → 25428 taken 3 times.
✗ Branch 25427 → 28699 not taken.
 3 punpcklwd(r1, zero);
2359
1/6
✗ Branch 420 → 421 not taken.
✗ Branch 420 → 3585 not taken.
✗ Branch 21078 → 21079 not taken.
✗ Branch 21078 → 24243 not taken.
✓ Branch 25428 → 25429 taken 3 times.
✗ Branch 25428 → 28699 not taken.
 3 cvtdq2ps(r1, r1);
2360
1/2
✓ Branch 25429 → 25430 taken 3 times.
✗ Branch 25429 → 25451 not taken.
 3 if (maskIt)
2361
1/4
✓ Branch 25430 → 25431 taken 3 times.
✗ Branch 25430 → 25435 not taken.
✗ Branch 25430 → 25439 not taken.
✗ Branch 25430 → 25443 not taken.
 3 doMask(r1, constptr, planewidth);
2362
1/6
✗ Branch 421 → 422 not taken.
✗ Branch 421 → 3585 not taken.
✗ Branch 21079 → 21080 not taken.
✗ Branch 21079 → 24243 not taken.
✓ Branch 25451 → 25452 taken 3 times.
✗ Branch 25451 → 28699 not taken.
 3 stack1.push_back(r1);
2363 }
2364 else {
2365
2/8
✓ Branch 4974 → 4975 taken 3 times.
✗ Branch 4974 → 10961 not taken.
✓ Branch 4975 → 4976 taken 3 times.
✗ Branch 4975 → 10961 not taken.
✗ Branch 13112 → 13113 not taken.
✗ Branch 13112 → 19206 not taken.
✗ Branch 13113 → 13114 not taken.
✗ Branch 13113 → 19206 not taken.
 3 XmmReg r1, r2;
2366
1/4
✓ Branch 4976 → 4977 taken 3 times.
✗ Branch 4976 → 10961 not taken.
✗ Branch 13114 → 13115 not taken.
✗ Branch 13114 → 19206 not taken.
 3 Reg a;
2367
2/8
✓ Branch 4978 → 4979 taken 3 times.
✗ Branch 4978 → 10956 not taken.
✓ Branch 4979 → 4980 taken 3 times.
✗ Branch 4979 → 10956 not taken.
✗ Branch 13116 → 13117 not taken.
✗ Branch 13116 → 19201 not taken.
✗ Branch 13117 → 13118 not taken.
✗ Branch 13117 → 19201 not taken.
 3 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2368
2/8
✓ Branch 4981 → 4982 taken 3 times.
✗ Branch 4981 → 10958 not taken.
✓ Branch 4982 → 4983 taken 3 times.
✗ Branch 4982 → 10958 not taken.
✗ Branch 13119 → 13120 not taken.
✗ Branch 13119 → 19203 not taken.
✗ Branch 13120 → 13121 not taken.
✗ Branch 13120 → 19203 not taken.
 3 movdqa(r1, xmmword_ptr[a]);
2369
1/4
✓ Branch 4983 → 4984 taken 3 times.
✗ Branch 4983 → 10961 not taken.
✗ Branch 13121 → 13122 not taken.
✗ Branch 13121 → 19206 not taken.
 3 movdqa(r2, r1);
2370
1/4
✓ Branch 4984 → 4985 taken 3 times.
✗ Branch 4984 → 10961 not taken.
✗ Branch 13122 → 13123 not taken.
✗ Branch 13122 → 19206 not taken.
 3 punpcklwd(r1, zero);
2371
1/4
✓ Branch 4985 → 4986 taken 3 times.
✗ Branch 4985 → 10961 not taken.
✗ Branch 13123 → 13124 not taken.
✗ Branch 13123 → 19206 not taken.
 3 punpckhwd(r2, zero);
2372
1/4
✓ Branch 4986 → 4987 taken 3 times.
✗ Branch 4986 → 10961 not taken.
✗ Branch 13124 → 13125 not taken.
✗ Branch 13124 → 19206 not taken.
 3 cvtdq2ps(r1, r1);
2373
1/4
✓ Branch 4987 → 4988 taken 3 times.
✗ Branch 4987 → 10961 not taken.
✗ Branch 13125 → 13126 not taken.
✗ Branch 13125 → 19206 not taken.
 3 cvtdq2ps(r2, r2);
2374 ✗ if (maskIt)
2375 ✗ doMask(r2, constptr, planewidth);
2376
2/8
✓ Branch 4988 → 4989 taken 3 times.
✗ Branch 4988 → 10960 not taken.
✓ Branch 4989 → 4990 taken 3 times.
✗ Branch 4989 → 10960 not taken.
✗ Branch 13148 → 13149 not taken.
✗ Branch 13148 → 19205 not taken.
✗ Branch 13149 → 13150 not taken.
✗ Branch 13149 → 19205 not taken.
 3 stack.push_back(std::make_pair(r1, r2));
2377 }
2378 }
2379
4/10
✗ Branch 423 → 424 not taken.
✗ Branch 423 → 435 not taken.
✓ Branch 4991 → 4992 taken 1 time.
✓ Branch 4991 → 5008 taken 18 times.
✗ Branch 13151 → 13152 not taken.
✓ Branch 13151 → 13189 taken 7 times.
✗ Branch 21081 → 21082 not taken.
✗ Branch 21081 → 21093 not taken.
✗ Branch 25453 → 25454 not taken.
✓ Branch 25453 → 25486 taken 9 times.
 35 else if (iter.op == opLoadSrcF32) {
2380 if (processSingle) {
2381 ✗ XmmReg r1;
2382 ✗ Reg a;
2383 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2384 ✗ movdqa(r1, xmmword_ptr[a]);
2385 ✗ if (maskIt)
2386 ✗ doMask(r1, constptr, planewidth);
2387 ✗ stack1.push_back(r1);
2388 }
2389 else {
2390
2/8
✓ Branch 4992 → 4993 taken 1 time.
✗ Branch 4992 → 10969 not taken.
✓ Branch 4993 → 4994 taken 1 time.
✗ Branch 4993 → 10969 not taken.
✗ Branch 13152 → 13153 not taken.
✗ Branch 13152 → 19214 not taken.
✗ Branch 13153 → 13154 not taken.
✗ Branch 13153 → 19214 not taken.
 1 XmmReg r1, r2;
2391
1/4
✓ Branch 4994 → 4995 taken 1 time.
✗ Branch 4994 → 10969 not taken.
✗ Branch 13154 → 13155 not taken.
✗ Branch 13154 → 19214 not taken.
 1 Reg a;
2392
2/8
✓ Branch 4996 → 4997 taken 1 time.
✗ Branch 4996 → 10962 not taken.
✓ Branch 4997 → 4998 taken 1 time.
✗ Branch 4997 → 10962 not taken.
✗ Branch 13156 → 13157 not taken.
✗ Branch 13156 → 19207 not taken.
✗ Branch 13157 → 13158 not taken.
✗ Branch 13157 → 19207 not taken.
 1 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2393
2/8
✓ Branch 4999 → 5000 taken 1 time.
✗ Branch 4999 → 10964 not taken.
✓ Branch 5000 → 5001 taken 1 time.
✗ Branch 5000 → 10964 not taken.
✗ Branch 13159 → 13160 not taken.
✗ Branch 13159 → 19209 not taken.
✗ Branch 13160 → 13161 not taken.
✗ Branch 13160 → 19209 not taken.
 1 movdqa(r1, xmmword_ptr[a]);
2394
2/8
✓ Branch 5002 → 5003 taken 1 time.
✗ Branch 5002 → 10966 not taken.
✓ Branch 5003 → 5004 taken 1 time.
✗ Branch 5003 → 10966 not taken.
✗ Branch 13162 → 13163 not taken.
✗ Branch 13162 → 19211 not taken.
✗ Branch 13163 → 13164 not taken.
✗ Branch 13163 → 19211 not taken.
 1 movdqa(r2, xmmword_ptr[a + 16]);
2395 ✗ if (maskIt)
2396 ✗ doMask(r2, constptr, planewidth);
2397
2/8
✓ Branch 5005 → 5006 taken 1 time.
✗ Branch 5005 → 10968 not taken.
✓ Branch 5006 → 5007 taken 1 time.
✗ Branch 5006 → 10968 not taken.
✗ Branch 13186 → 13187 not taken.
✗ Branch 13186 → 19213 not taken.
✗ Branch 13187 → 13188 not taken.
✗ Branch 13187 → 19213 not taken.
 1 stack.push_back(std::make_pair(r1, r2));
2398 }
2399 }
2400
3/10
✗ Branch 435 → 436 not taken.
✗ Branch 435 → 447 not taken.
✗ Branch 5008 → 5009 not taken.
✓ Branch 5008 → 5025 taken 18 times.
✗ Branch 13189 → 13190 not taken.
✓ Branch 13189 → 13227 taken 7 times.
✗ Branch 21093 → 21094 not taken.
✗ Branch 21093 → 21105 not taken.
✗ Branch 25486 → 25487 not taken.
✓ Branch 25486 → 25519 taken 9 times.
 34 else if (iter.op == opLoadSrcF16) { // not supported in avs+
2401 if (processSingle) {
2402 ✗ XmmReg r1;
2403 ✗ Reg a;
2404 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2405 ✗ vcvtph2ps(r1, qword_ptr[a]);
2406 ✗ if (maskIt)
2407 ✗ doMask(r1, constptr, planewidth);
2408 ✗ stack1.push_back(r1);
2409 }
2410 else {
2411 ✗ XmmReg r1, r2;
2412 ✗ Reg a;
2413 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
2414 ✗ vcvtph2ps(r1, qword_ptr[a]);
2415 ✗ vcvtph2ps(r2, qword_ptr[a + 8]);
2416 ✗ if (maskIt)
2417 ✗ doMask(r2, constptr, planewidth);
2418 ✗ stack.push_back(std::make_pair(r1, r2));
2419 }
2420 }
2421
3/10
✗ Branch 447 → 448 not taken.
✗ Branch 447 → 455 not taken.
✗ Branch 5025 → 5026 not taken.
✓ Branch 5025 → 5039 taken 18 times.
✗ Branch 13227 → 13228 not taken.
✓ Branch 13227 → 13262 taken 7 times.
✗ Branch 21105 → 21106 not taken.
✗ Branch 21105 → 21113 not taken.
✗ Branch 25519 → 25520 not taken.
✓ Branch 25519 → 25548 taken 9 times.
 34 else if (iter.op == opLoadVar) {
2422 if (processSingle) {
2423 ✗ XmmReg r1;
2424 // 16 bytes/variable
2425 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 16 * iter.e.ival;
2426 ✗ movdqa(r1, xmmword_ptr[regptrs + offset]);
2427 ✗ if (maskIt)
2428 ✗ doMask(r1, constptr, planewidth);
2429 ✗ stack1.push_back(r1);
2430 }
2431 else {
2432 ✗ XmmReg r1, r2;
2433 // 32 bytes/variable
2434 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 32 * iter.e.ival;
2435 ✗ movdqa(r1, xmmword_ptr[regptrs + offset]);
2436 ✗ movdqa(r2, xmmword_ptr[regptrs + offset + 16]);
2437 ✗ if (maskIt)
2438 ✗ doMask(r2, constptr, planewidth);
2439 ✗ stack.push_back(std::make_pair(r1, r2));
2440 }
2441 }
2442
5/10
✗ Branch 455 → 456 not taken.
✗ Branch 455 → 466 not taken.
✓ Branch 5039 → 5040 taken 3 times.
✓ Branch 5039 → 5052 taken 15 times.
✗ Branch 13262 → 13263 not taken.
✓ Branch 13262 → 13297 taken 7 times.
✗ Branch 21113 → 21114 not taken.
✗ Branch 21113 → 21124 not taken.
✓ Branch 25548 → 25549 taken 3 times.
✓ Branch 25548 → 25580 taken 6 times.
 34 else if (iter.op == opLoadConst) {
2443 if (processSingle) {
2444
1/6
✗ Branch 456 → 457 not taken.
✗ Branch 456 → 3601 not taken.
✗ Branch 21114 → 21115 not taken.
✗ Branch 21114 → 24259 not taken.
✓ Branch 25549 → 25550 taken 3 times.
✗ Branch 25549 → 28715 not taken.
 3 XmmReg r1;
2445
1/6
✗ Branch 457 → 458 not taken.
✗ Branch 457 → 3601 not taken.
✗ Branch 21115 → 21116 not taken.
✗ Branch 21115 → 24259 not taken.
✓ Branch 25550 → 25551 taken 3 times.
✗ Branch 25550 → 28715 not taken.
 3 Reg a;
2446
2/12
✗ Branch 458 → 459 not taken.
✗ Branch 458 → 3599 not taken.
✗ Branch 459 → 460 not taken.
✗ Branch 459 → 3599 not taken.
✗ Branch 21116 → 21117 not taken.
✗ Branch 21116 → 24257 not taken.
✗ Branch 21117 → 21118 not taken.
✗ Branch 21117 → 24257 not taken.
✓ Branch 25551 → 25552 taken 3 times.
✗ Branch 25551 → 28713 not taken.
✓ Branch 25552 → 25553 taken 3 times.
✗ Branch 25552 → 28713 not taken.
 3 mov(a, iter.e.ival);
2447
1/6
✗ Branch 460 → 461 not taken.
✗ Branch 460 → 3601 not taken.
✗ Branch 21118 → 21119 not taken.
✗ Branch 21118 → 24259 not taken.
✓ Branch 25553 → 25554 taken 3 times.
✗ Branch 25553 → 28715 not taken.
 3 movd(r1, a);
2448
2/12
✗ Branch 461 → 462 not taken.
✗ Branch 461 → 3600 not taken.
✗ Branch 462 → 463 not taken.
✗ Branch 462 → 3600 not taken.
✗ Branch 21119 → 21120 not taken.
✗ Branch 21119 → 24258 not taken.
✗ Branch 21120 → 21121 not taken.
✗ Branch 21120 → 24258 not taken.
✓ Branch 25554 → 25555 taken 3 times.
✗ Branch 25554 → 28714 not taken.
✓ Branch 25555 → 25556 taken 3 times.
✗ Branch 25555 → 28714 not taken.
 3 shufps(r1, r1, 0);
2449
1/2
✓ Branch 25556 → 25557 taken 3 times.
✗ Branch 25556 → 25578 not taken.
 3 if (maskIt)
2450
1/4
✓ Branch 25557 → 25558 taken 3 times.
✗ Branch 25557 → 25562 not taken.
✗ Branch 25557 → 25566 not taken.
✗ Branch 25557 → 25570 not taken.
 3 doMask(r1, constptr, planewidth);
2451
1/6
✗ Branch 464 → 465 not taken.
✗ Branch 464 → 3601 not taken.
✗ Branch 21122 → 21123 not taken.
✗ Branch 21122 → 24259 not taken.
✓ Branch 25578 → 25579 taken 3 times.
✗ Branch 25578 → 28715 not taken.
 3 stack1.push_back(r1);
2452 }
2453 else {
2454
2/8
✓ Branch 5040 → 5041 taken 3 times.
✗ Branch 5040 → 10988 not taken.
✓ Branch 5041 → 5042 taken 3 times.
✗ Branch 5041 → 10988 not taken.
✗ Branch 13263 → 13264 not taken.
✗ Branch 13263 → 19233 not taken.
✗ Branch 13264 → 13265 not taken.
✗ Branch 13264 → 19233 not taken.
 3 XmmReg r1, r2;
2455
1/4
✓ Branch 5042 → 5043 taken 3 times.
✗ Branch 5042 → 10988 not taken.
✗ Branch 13265 → 13266 not taken.
✗ Branch 13265 → 19233 not taken.
 3 Reg a;
2456
2/8
✓ Branch 5043 → 5044 taken 3 times.
✗ Branch 5043 → 10985 not taken.
✓ Branch 5044 → 5045 taken 3 times.
✗ Branch 5044 → 10985 not taken.
✗ Branch 13266 → 13267 not taken.
✗ Branch 13266 → 19230 not taken.
✗ Branch 13267 → 13268 not taken.
✗ Branch 13267 → 19230 not taken.
 3 mov(a, iter.e.ival);
2457
1/4
✓ Branch 5045 → 5046 taken 3 times.
✗ Branch 5045 → 10988 not taken.
✗ Branch 13268 → 13269 not taken.
✗ Branch 13268 → 19233 not taken.
 3 movd(r1, a);
2458
2/8
✓ Branch 5046 → 5047 taken 3 times.
✗ Branch 5046 → 10986 not taken.
✓ Branch 5047 → 5048 taken 3 times.
✗ Branch 5047 → 10986 not taken.
✗ Branch 13269 → 13270 not taken.
✗ Branch 13269 → 19231 not taken.
✗ Branch 13270 → 13271 not taken.
✗ Branch 13270 → 19231 not taken.
 3 shufps(r1, r1, 0);
2459
1/4
✓ Branch 5048 → 5049 taken 3 times.
✗ Branch 5048 → 10988 not taken.
✗ Branch 13271 → 13272 not taken.
✗ Branch 13271 → 19233 not taken.
 3 movaps(r2, r1);
2460 ✗ if (maskIt)
2461 ✗ doMask(r2, constptr, planewidth);
2462
2/8
✓ Branch 5049 → 5050 taken 3 times.
✗ Branch 5049 → 10987 not taken.
✓ Branch 5050 → 5051 taken 3 times.
✗ Branch 5050 → 10987 not taken.
✗ Branch 13294 → 13295 not taken.
✗ Branch 13294 → 19232 not taken.
✗ Branch 13295 → 13296 not taken.
✗ Branch 13295 → 19232 not taken.
 3 stack.push_back(std::make_pair(r1, r2));
2463 }
2464 }
2465
3/10
✗ Branch 466 → 467 not taken.
✗ Branch 466 → 479 not taken.
✗ Branch 5052 → 5053 not taken.
✓ Branch 5052 → 5069 taken 15 times.
✗ Branch 13297 → 13298 not taken.
✓ Branch 13297 → 13314 taken 7 times.
✗ Branch 21124 → 21125 not taken.
✗ Branch 21124 → 21137 not taken.
✗ Branch 25580 → 25581 not taken.
✓ Branch 25580 → 25593 taken 6 times.
 28 else if (iter.op == opDup) {
2466 if (processSingle) {
2467 ✗ auto p = std::next(stack1.rbegin(), iter.e.ival);
2468 ✗ XmmReg r1;
2469 ✗ movaps(r1, *p);
2470 ✗ stack1.push_back(r1);
2471 }
2472 else {
2473 ✗ auto p = std::next(stack.rbegin(), iter.e.ival);
2474 ✗ XmmReg r1, r2;
2475 ✗ movaps(r1, p->first);
2476 ✗ movaps(r2, p->second);
2477 ✗ stack.push_back(std::make_pair(r1, r2));
2478 }
2479 }
2480
3/10
✗ Branch 479 → 480 not taken.
✗ Branch 479 → 491 not taken.
✗ Branch 5069 → 5070 not taken.
✓ Branch 5069 → 5081 taken 15 times.
✗ Branch 13314 → 13315 not taken.
✓ Branch 13314 → 13326 taken 7 times.
✗ Branch 21137 → 21138 not taken.
✗ Branch 21137 → 21149 not taken.
✗ Branch 25593 → 25594 not taken.
✓ Branch 25593 → 25605 taken 6 times.
 28 else if (iter.op == opSwap) {
2481 if (processSingle) {
2482 ✗ std::swap(stack1.back(), *std::next(stack1.rbegin(), iter.e.ival));
2483 }
2484 else {
2485 ✗ std::swap(stack.back(), *std::next(stack.rbegin(), iter.e.ival));
2486 }
2487 }
2488
6/10
✗ Branch 491 → 492 not taken.
✗ Branch 491 → 497 not taken.
✓ Branch 5081 → 5082 taken 7 times.
✓ Branch 5081 → 5088 taken 8 times.
✓ Branch 13326 → 13327 taken 4 times.
✓ Branch 13326 → 13333 taken 3 times.
✗ Branch 21149 → 21150 not taken.
✗ Branch 21149 → 21155 not taken.
✓ Branch 25605 → 25606 taken 3 times.
✓ Branch 25605 → 25611 taken 3 times.
 28 else if (iter.op == opAdd) {
2489 if (processSingle) {
2490
1/6
✗ Branch 495 → 496 not taken.
✗ Branch 495 → 3606 not taken.
✗ Branch 21153 → 21154 not taken.
✗ Branch 21153 → 24264 not taken.
✓ Branch 25609 → 25610 taken 3 times.
✗ Branch 25609 → 28720 not taken.
 3 TwoArgOp_Single(addps)
2491 }
2492 else {
2493
4/8
✓ Branch 5085 → 5086 taken 7 times.
✗ Branch 5085 → 10994 not taken.
✓ Branch 5086 → 5087 taken 7 times.
✗ Branch 5086 → 10994 not taken.
✓ Branch 13330 → 13331 taken 4 times.
✗ Branch 13330 → 19239 not taken.
✓ Branch 13331 → 13332 taken 4 times.
✗ Branch 13331 → 19239 not taken.
 11 TwoArgOp(addps)
2494 }
2495 }
2496
3/10
✗ Branch 497 → 498 not taken.
✗ Branch 497 → 503 not taken.
✗ Branch 5088 → 5089 not taken.
✓ Branch 5088 → 5095 taken 8 times.
✗ Branch 13333 → 13334 not taken.
✓ Branch 13333 → 13340 taken 3 times.
✗ Branch 21155 → 21156 not taken.
✗ Branch 21155 → 21161 not taken.
✗ Branch 25611 → 25612 not taken.
✓ Branch 25611 → 25617 taken 3 times.
 14 else if (iter.op == opSub) {
2497 if (processSingle) {
2498 ✗ TwoArgOp_Single(subps)
2499 }
2500 else {
2501 ✗ TwoArgOp(subps)
2502 }
2503 }
2504
3/10
✗ Branch 503 → 504 not taken.
✗ Branch 503 → 509 not taken.
✗ Branch 5095 → 5096 not taken.
✓ Branch 5095 → 5102 taken 8 times.
✗ Branch 13340 → 13341 not taken.
✓ Branch 13340 → 13347 taken 3 times.
✗ Branch 21161 → 21162 not taken.
✗ Branch 21161 → 21167 not taken.
✗ Branch 25617 → 25618 not taken.
✓ Branch 25617 → 25623 taken 3 times.
 14 else if (iter.op == opMul) {
2505 if (processSingle) {
2506 ✗ TwoArgOp_Single(mulps)
2507 }
2508 else {
2509 ✗ TwoArgOp(mulps)
2510 }
2511 }
2512
3/10
✗ Branch 509 → 510 not taken.
✗ Branch 509 → 515 not taken.
✗ Branch 5102 → 5103 not taken.
✓ Branch 5102 → 5109 taken 8 times.
✗ Branch 13347 → 13348 not taken.
✓ Branch 13347 → 13354 taken 3 times.
✗ Branch 21167 → 21168 not taken.
✗ Branch 21167 → 21173 not taken.
✗ Branch 25623 → 25624 not taken.
✓ Branch 25623 → 25629 taken 3 times.
 14 else if (iter.op == opDiv) {
2513 if (processSingle) {
2514 ✗ TwoArgOp_Single(divps)
2515 }
2516 else {
2517 ✗ TwoArgOp(divps)
2518 }
2519 }
2520
3/10
✗ Branch 515 → 516 not taken.
✗ Branch 515 → 527 not taken.
✗ Branch 5109 → 5110 not taken.
✓ Branch 5109 → 5128 taken 8 times.
✗ Branch 13354 → 13355 not taken.
✓ Branch 13354 → 13373 taken 3 times.
✗ Branch 21173 → 21174 not taken.
✗ Branch 21173 → 21185 not taken.
✗ Branch 25629 → 25630 not taken.
✓ Branch 25629 → 25641 taken 3 times.
 14 else if (iter.op == opFmod) {
2521 if (processSingle) {
2522 ✗ auto t1 = stack1.back(); // despite the Intel compiler warnings, this is intentionally a copy and not a reference (valid for all such messages as well)
2523 ✗ stack1.pop_back();
2524 ✗ auto &t2 = stack1.back();
2525 ✗ FMOD_PS(t2, t1)
2526 }
2527 else {
2528 ✗ auto t1 = stack.back();
2529 ✗ stack.pop_back();
2530 ✗ auto &t2 = stack.back();
2531 ✗ FMOD_PS(t2.first, t1.first)
2532 ✗ FMOD_PS(t2.second, t1.second)
2533 }
2534 }
2535
3/10
✗ Branch 527 → 528 not taken.
✗ Branch 527 → 533 not taken.
✗ Branch 5128 → 5129 not taken.
✓ Branch 5128 → 5135 taken 8 times.
✗ Branch 13373 → 13374 not taken.
✓ Branch 13373 → 13380 taken 3 times.
✗ Branch 21185 → 21186 not taken.
✗ Branch 21185 → 21191 not taken.
✗ Branch 25641 → 25642 not taken.
✓ Branch 25641 → 25647 taken 3 times.
 14 else if (iter.op == opMax) {
2536 if (processSingle) {
2537 ✗ TwoArgOp_Single(maxps)
2538 }
2539 else {
2540 ✗ TwoArgOp(maxps)
2541 }
2542 }
2543
3/10
✗ Branch 533 → 534 not taken.
✗ Branch 533 → 539 not taken.
✗ Branch 5135 → 5136 not taken.
✓ Branch 5135 → 5142 taken 8 times.
✗ Branch 13380 → 13381 not taken.
✓ Branch 13380 → 13387 taken 3 times.
✗ Branch 21191 → 21192 not taken.
✗ Branch 21191 → 21197 not taken.
✗ Branch 25647 → 25648 not taken.
✓ Branch 25647 → 25653 taken 3 times.
 14 else if (iter.op == opMin) {
2544 if (processSingle) {
2545 ✗ TwoArgOp_Single(minps)
2546 }
2547 else {
2548 ✗ TwoArgOp(minps)
2549 }
2550 }
2551
4/10
✗ Branch 539 → 540 not taken.
✗ Branch 539 → 543 not taken.
✓ Branch 5142 → 5143 taken 1 time.
✓ Branch 5142 → 5148 taken 7 times.
✗ Branch 13387 → 13388 not taken.
✓ Branch 13387 → 13393 taken 3 times.
✗ Branch 21197 → 21198 not taken.
✗ Branch 21197 → 21201 not taken.
✗ Branch 25653 → 25654 not taken.
✓ Branch 25653 → 25657 taken 3 times.
 14 else if (iter.op == opSqrt) {
2552 if (processSingle) {
2553 ✗ auto &t1 = stack1.back();
2554 ✗ maxps(t1, zero);
2555 ✗ sqrtps(t1, t1);
2556 }
2557 else {
2558 1 auto &t1 = stack.back();
2559
1/4
✓ Branch 5144 → 5145 taken 1 time.
✗ Branch 5144 → 12428 not taken.
✗ Branch 13389 → 13390 not taken.
✗ Branch 13389 → 20673 not taken.
 1 maxps(t1.first, zero);
2560
1/4
✓ Branch 5145 → 5146 taken 1 time.
✗ Branch 5145 → 12428 not taken.
✗ Branch 13390 → 13391 not taken.
✗ Branch 13390 → 20673 not taken.
 1 maxps(t1.second, zero);
2561
1/4
✓ Branch 5146 → 5147 taken 1 time.
✗ Branch 5146 → 12428 not taken.
✗ Branch 13391 → 13392 not taken.
✗ Branch 13391 → 20673 not taken.
 1 sqrtps(t1.first, t1.first);
2562
1/4
✓ Branch 5147 → 10684 taken 1 time.
✗ Branch 5147 → 12428 not taken.
✗ Branch 13392 → 18929 not taken.
✗ Branch 13392 → 20673 not taken.
 1 sqrtps(t1.second, t1.second);
2563 }
2564 }
2565 // Integer store operations: Why sometimes C version differs from SSE: convert to int from .5 intermediates.
2566 // Simd version of float -> int32 (cvtps2dq) is using the SSE rounding mode "round to nearest"
2567 // C version is using: (uint8_t)(f + 0.5f) which turnes into cvttps, typecast to int uses trunc
2568 // Even for positive numbers they are not the same, when converting occurs exactly from the halfway
2569 // SSE is using Banker's rounding, which rounds to the nearest _even_ integer value. https://en.wikipedia.org/wiki/IEEE_754#Roundings_to_nearest
2570 // C SSE
2571 // 0.5 1 0
2572 // 1.5 2 2
2573 // 2.5 3 2
2574 // 3.5 4 4
2575 // 3.7.1test27: no more banker's rounding. Using cvttps and +0.5f
2576
4/10
✗ Branch 543 → 544 not taken.
✗ Branch 543 → 565 not taken.
✓ Branch 5148 → 5149 taken 3 times.
✓ Branch 5148 → 5179 taken 4 times.
✓ Branch 13393 → 13394 taken 3 times.
✗ Branch 13393 → 13424 not taken.
✗ Branch 21201 → 21202 not taken.
✗ Branch 21201 → 21223 not taken.
✗ Branch 25657 → 25658 not taken.
✓ Branch 25657 → 25679 taken 3 times.
 13 else if (iter.op == opStore8) {
2577 if (processSingle) {
2578 ✗ auto t1 = stack1.back();
2579 ✗ stack1.pop_back();
2580 ✗ XmmReg r1;
2581 ✗ Reg a;
2582 ✗ addps(t1, CPTR(elfloat_half)); // rounder for truncate! no banker's rounding
2583 ✗ maxps(t1, zero);
2584 ✗ minps(t1, CPTR(elstore8));
2585 ✗ mov(a, ptr[regptrs]);
2586 ✗ cvttps2dq(t1, t1); // 00 w3 00 w2 00 w1 00 w0 -- min/max clamp ensures that high words are zero
2587 ✗ packssdw(t1, zero); // _mm_packs_epi32: w7 w6 w5 w4 w3 w2 w1 w0
2588 ✗ packuswb(t1, zero); // _mm_packus_epi16: 0 0 0 0 0 0 0 0 b7 b6 b5 b4 b3 b2 b1 b0
2589 ✗ movd(dword_ptr[a], t1);
2590 }
2591 else {
2592 6 auto t1 = stack.back();
2593 6 stack.pop_back();
2594
4/8
✓ Branch 5151 → 5152 taken 3 times.
✗ Branch 5151 → 11015 not taken.
✓ Branch 5152 → 5153 taken 3 times.
✗ Branch 5152 → 11015 not taken.
✓ Branch 13396 → 13397 taken 3 times.
✗ Branch 13396 → 19260 not taken.
✓ Branch 13397 → 13398 taken 3 times.
✗ Branch 13397 → 19260 not taken.
 6 XmmReg r1, r2;
2595
2/4
✓ Branch 5153 → 5154 taken 3 times.
✗ Branch 5153 → 11015 not taken.
✓ Branch 13398 → 13399 taken 3 times.
✗ Branch 13398 → 19260 not taken.
 6 Reg a;
2596
4/8
✓ Branch 5155 → 5156 taken 3 times.
✗ Branch 5155 → 11003 not taken.
✓ Branch 5156 → 5157 taken 3 times.
✗ Branch 5156 → 11003 not taken.
✓ Branch 13400 → 13401 taken 3 times.
✗ Branch 13400 → 19248 not taken.
✓ Branch 13401 → 13402 taken 3 times.
✗ Branch 13401 → 19248 not taken.
 6 addps(t1.first, CPTR(elfloat_half)); // rounder for truncate! no banker's rounding
2597
2/4
✓ Branch 5157 → 5158 taken 3 times.
✗ Branch 5157 → 11015 not taken.
✓ Branch 13402 → 13403 taken 3 times.
✗ Branch 13402 → 19260 not taken.
 6 maxps(t1.first, zero);
2598
4/8
✓ Branch 5159 → 5160 taken 3 times.
✗ Branch 5159 → 11005 not taken.
✓ Branch 5160 → 5161 taken 3 times.
✗ Branch 5160 → 11005 not taken.
✓ Branch 13404 → 13405 taken 3 times.
✗ Branch 13404 → 19250 not taken.
✓ Branch 13405 → 13406 taken 3 times.
✗ Branch 13405 → 19250 not taken.
 6 addps(t1.second, CPTR(elfloat_half)); // rounder for truncate! no banker's rounding
2599
2/4
✓ Branch 5161 → 5162 taken 3 times.
✗ Branch 5161 → 11015 not taken.
✓ Branch 13406 → 13407 taken 3 times.
✗ Branch 13406 → 19260 not taken.
 6 maxps(t1.second, zero);
2600
4/8
✓ Branch 5163 → 5164 taken 3 times.
✗ Branch 5163 → 11007 not taken.
✓ Branch 5164 → 5165 taken 3 times.
✗ Branch 5164 → 11007 not taken.
✓ Branch 13408 → 13409 taken 3 times.
✗ Branch 13408 → 19252 not taken.
✓ Branch 13409 → 13410 taken 3 times.
✗ Branch 13409 → 19252 not taken.
 6 minps(t1.first, CPTR(elstore8));
2601
4/8
✓ Branch 5166 → 5167 taken 3 times.
✗ Branch 5166 → 11009 not taken.
✓ Branch 5167 → 5168 taken 3 times.
✗ Branch 5167 → 11009 not taken.
✓ Branch 13411 → 13412 taken 3 times.
✗ Branch 13411 → 19254 not taken.
✓ Branch 13412 → 13413 taken 3 times.
✗ Branch 13412 → 19254 not taken.
 6 minps(t1.second, CPTR(elstore8));
2602
4/8
✓ Branch 5169 → 5170 taken 3 times.
✗ Branch 5169 → 11011 not taken.
✓ Branch 5170 → 5171 taken 3 times.
✗ Branch 5170 → 11011 not taken.
✓ Branch 13414 → 13415 taken 3 times.
✗ Branch 13414 → 19256 not taken.
✓ Branch 13415 → 13416 taken 3 times.
✗ Branch 13415 → 19256 not taken.
 6 mov(a, ptr[regptrs]);
2603
2/4
✓ Branch 5171 → 5172 taken 3 times.
✗ Branch 5171 → 11015 not taken.
✓ Branch 13416 → 13417 taken 3 times.
✗ Branch 13416 → 19260 not taken.
 6 cvttps2dq(t1.first, t1.first); // 00 w3 00 w2 00 w1 00 w0 -- min/max clamp ensures that high words are zero
2604
2/4
✓ Branch 5172 → 5173 taken 3 times.
✗ Branch 5172 → 11015 not taken.
✓ Branch 13417 → 13418 taken 3 times.
✗ Branch 13417 → 19260 not taken.
 6 cvttps2dq(t1.second, t1.second); // 00 w7 00 w6 00 w5 00 w4
2605 // t1.second is the lo
2606
2/4
✓ Branch 5173 → 5174 taken 3 times.
✗ Branch 5173 → 11015 not taken.
✓ Branch 13418 → 13419 taken 3 times.
✗ Branch 13418 → 19260 not taken.
 6 packssdw(t1.first, t1.second); // _mm_packs_epi32: w7 w6 w5 w4 w3 w2 w1 w0
2607
2/4
✓ Branch 5174 → 5175 taken 3 times.
✗ Branch 5174 → 11015 not taken.
✓ Branch 13419 → 13420 taken 3 times.
✗ Branch 13419 → 19260 not taken.
 6 packuswb(t1.first, zero); // _mm_packus_epi16: 0 0 0 0 0 0 0 0 b7 b6 b5 b4 b3 b2 b1 b0
2608
4/8
✓ Branch 5176 → 5177 taken 3 times.
✗ Branch 5176 → 11013 not taken.
✓ Branch 5177 → 5178 taken 3 times.
✗ Branch 5177 → 11013 not taken.
✓ Branch 13421 → 13422 taken 3 times.
✗ Branch 13421 → 19258 not taken.
✓ Branch 13422 → 13423 taken 3 times.
✗ Branch 13422 → 19258 not taken.
 6 movq(mmword_ptr[a], t1.first);
2609 }
2610 }
2611
2/10
✗ Branch 565 → 566 not taken.
✗ Branch 565 → 569 not taken.
✓ Branch 5179 → 5180 taken 4 times.
✗ Branch 5179 → 5183 not taken.
✗ Branch 13424 → 13425 not taken.
✗ Branch 13424 → 13428 not taken.
✗ Branch 21223 → 21224 not taken.
✗ Branch 21223 → 21227 not taken.
✓ Branch 25679 → 25680 taken 3 times.
✗ Branch 25679 → 25683 not taken.
 7 else if (iter.op == opStore10 // avs+
2612
2/10
✗ Branch 566 → 567 not taken.
✗ Branch 566 → 569 not taken.
✓ Branch 5180 → 5181 taken 4 times.
✗ Branch 5180 → 5183 not taken.
✗ Branch 13425 → 13426 not taken.
✗ Branch 13425 → 13428 not taken.
✗ Branch 21224 → 21225 not taken.
✗ Branch 21224 → 21227 not taken.
✓ Branch 25680 → 25681 taken 3 times.
✗ Branch 25680 → 25683 not taken.
 7 || iter.op == opStore12 // avs+
2613
2/10
✗ Branch 567 → 568 not taken.
✗ Branch 567 → 569 not taken.
✓ Branch 5181 → 5182 taken 4 times.
✗ Branch 5181 → 5183 not taken.
✗ Branch 13426 → 13427 not taken.
✗ Branch 13426 → 13428 not taken.
✗ Branch 21225 → 21226 not taken.
✗ Branch 21225 → 21227 not taken.
✓ Branch 25681 → 25682 taken 3 times.
✗ Branch 25681 → 25683 not taken.
 7 || iter.op == opStore14 // avs+
2614
3/10
✗ Branch 568 → 569 not taken.
✗ Branch 568 → 616 not taken.
✓ Branch 5182 → 5183 taken 3 times.
✓ Branch 5182 → 5253 taken 1 time.
✗ Branch 13427 → 13428 not taken.
✗ Branch 13427 → 13498 not taken.
✗ Branch 21226 → 21227 not taken.
✗ Branch 21226 → 21274 not taken.
✓ Branch 25682 → 25683 taken 3 times.
✗ Branch 25682 → 25730 not taken.
 7 || iter.op == opStore16
2615 ) {
2616 if (processSingle) {
2617 3 auto t1 = stack1.back();
2618 3 stack1.pop_back();
2619
1/6
✗ Branch 571 → 572 not taken.
✗ Branch 571 → 3639 not taken.
✗ Branch 21229 → 21230 not taken.
✗ Branch 21229 → 24297 not taken.
✓ Branch 25685 → 25686 taken 3 times.
✗ Branch 25685 → 28753 not taken.
 3 XmmReg r1;
2620
1/6
✗ Branch 572 → 573 not taken.
✗ Branch 572 → 3639 not taken.
✗ Branch 21230 → 21231 not taken.
✗ Branch 21230 → 24297 not taken.
✓ Branch 25686 → 25687 taken 3 times.
✗ Branch 25686 → 28753 not taken.
 3 Reg a;
2621
2/12
✗ Branch 574 → 575 not taken.
✗ Branch 574 → 3623 not taken.
✗ Branch 575 → 576 not taken.
✗ Branch 575 → 3623 not taken.
✗ Branch 21232 → 21233 not taken.
✗ Branch 21232 → 24281 not taken.
✗ Branch 21233 → 21234 not taken.
✗ Branch 21233 → 24281 not taken.
✓ Branch 25688 → 25689 taken 3 times.
✗ Branch 25688 → 28737 not taken.
✓ Branch 25689 → 25690 taken 3 times.
✗ Branch 25689 → 28737 not taken.
 3 addps(t1, CPTR(elfloat_half)); // rounder for truncate! no banker's rounding
2622
1/6
✗ Branch 576 → 577 not taken.
✗ Branch 576 → 3639 not taken.
✗ Branch 21234 → 21235 not taken.
✗ Branch 21234 → 24297 not taken.
✓ Branch 25690 → 25691 taken 3 times.
✗ Branch 25690 → 28753 not taken.
 3 maxps(t1, zero);
2623
1/15
✗ Branch 577 → 578 not taken.
✗ Branch 577 → 582 not taken.
✗ Branch 577 → 586 not taken.
✗ Branch 577 → 590 not taken.
✗ Branch 577 → 594 not taken.
✗ Branch 21235 → 21236 not taken.
✗ Branch 21235 → 21240 not taken.
✗ Branch 21235 → 21244 not taken.
✗ Branch 21235 → 21248 not taken.
✗ Branch 21235 → 21252 not taken.
✗ Branch 25691 → 25692 not taken.
✗ Branch 25691 → 25696 not taken.
✗ Branch 25691 → 25700 not taken.
✓ Branch 25691 → 25704 taken 3 times.
✗ Branch 25691 → 25708 not taken.
 3 switch (iter.op) {
2624 ✗ case opStore10:
2625 ✗ minps(t1, CPTR(elstore10));
2626 ✗ break;
2627 ✗ case opStore12:
2628 ✗ minps(t1, CPTR(elstore12));
2629 ✗ break;
2630 ✗ case opStore14:
2631 ✗ minps(t1, CPTR(elstore14));
2632 ✗ break;
2633 3 case opStore16:
2634
2/12
✗ Branch 591 → 592 not taken.
✗ Branch 591 → 3631 not taken.
✗ Branch 592 → 593 not taken.
✗ Branch 592 → 3631 not taken.
✗ Branch 21249 → 21250 not taken.
✗ Branch 21249 → 24289 not taken.
✗ Branch 21250 → 21251 not taken.
✗ Branch 21250 → 24289 not taken.
✓ Branch 25705 → 25706 taken 3 times.
✗ Branch 25705 → 28745 not taken.
✓ Branch 25706 → 25707 taken 3 times.
✗ Branch 25706 → 28745 not taken.
 3 minps(t1, CPTR(elstore16));
2635 3 break;
2636 }
2637
2/12
✗ Branch 595 → 596 not taken.
✗ Branch 595 → 3633 not taken.
✗ Branch 596 → 597 not taken.
✗ Branch 596 → 3633 not taken.
✗ Branch 21253 → 21254 not taken.
✗ Branch 21253 → 24291 not taken.
✗ Branch 21254 → 21255 not taken.
✗ Branch 21254 → 24291 not taken.
✓ Branch 25709 → 25710 taken 3 times.
✗ Branch 25709 → 28747 not taken.
✓ Branch 25710 → 25711 taken 3 times.
✗ Branch 25710 → 28747 not taken.
 3 mov(a, ptr[regptrs]);
2638
1/6
✗ Branch 597 → 598 not taken.
✗ Branch 597 → 3639 not taken.
✗ Branch 21255 → 21256 not taken.
✗ Branch 21255 → 24297 not taken.
✓ Branch 25711 → 25712 taken 3 times.
✗ Branch 25711 → 28753 not taken.
 3 cvttps2dq(t1, t1); // no cvtps, but cvttps
2639 // new
2640
1/9
✗ Branch 598 → 599 not taken.
✗ Branch 598 → 601 not taken.
✗ Branch 598 → 611 not taken.
✗ Branch 21256 → 21257 not taken.
✗ Branch 21256 → 21259 not taken.
✗ Branch 21256 → 21269 not taken.
✗ Branch 25712 → 25713 not taken.
✓ Branch 25712 → 25715 taken 3 times.
✗ Branch 25712 → 25725 not taken.
 3 switch (iter.op) {
2641 ✗ case opStore10:
2642 case opStore12:
2643 case opStore14:
2644 ✗ packssdw(t1, zero); // _mm_packs_epi32: w7 w6 w5 w4 w3 w2 w1 w0
2645 ✗ break;
2646 3 case opStore16:
2647
1/6
✗ Branch 601 → 602 not taken.
✗ Branch 601 → 603 not taken.
✗ Branch 21259 → 21260 not taken.
✗ Branch 21259 → 21261 not taken.
✓ Branch 25715 → 25716 taken 3 times.
✗ Branch 25715 → 25717 not taken.
 3 if (cpuFlags & CPUF_SSE4_1) {
2648
1/6
✗ Branch 602 → 610 not taken.
✗ Branch 602 → 3639 not taken.
✗ Branch 21260 → 21268 not taken.
✗ Branch 21260 → 24297 not taken.
✓ Branch 25716 → 25724 taken 3 times.
✗ Branch 25716 → 28753 not taken.
 3 packusdw(t1, zero); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
2649 }
2650 else {
2651 // old, sse2
2652 ✗ movdqa(r1, t1); // 00 w3 00 w2 00 w1 00 w0 -- min/max clamp ensures that high words are zero
2653 ✗ psrldq(t1, 6);
2654 ✗ por(t1, r1);
2655 ✗ pshuflw(t1, t1, 0b11011000);
2656 ✗ punpcklqdq(t1, zero);
2657 }
2658 3 break;
2659 }
2660
2/12
✗ Branch 612 → 613 not taken.
✗ Branch 612 → 3637 not taken.
✗ Branch 613 → 614 not taken.
✗ Branch 613 → 3637 not taken.
✗ Branch 21270 → 21271 not taken.
✗ Branch 21270 → 24295 not taken.
✗ Branch 21271 → 21272 not taken.
✗ Branch 21271 → 24295 not taken.
✓ Branch 25726 → 25727 taken 3 times.
✗ Branch 25726 → 28751 not taken.
✓ Branch 25727 → 25728 taken 3 times.
✗ Branch 25727 → 28751 not taken.
 3 movq(mmword_ptr[a], t1);
2661 }
2662 else {
2663 3 auto t1 = stack.back();
2664 3 stack.pop_back();
2665
2/8
✓ Branch 5185 → 5186 taken 3 times.
✗ Branch 5185 → 11044 not taken.
✓ Branch 5186 → 5187 taken 3 times.
✗ Branch 5186 → 11044 not taken.
✗ Branch 13430 → 13431 not taken.
✗ Branch 13430 → 19289 not taken.
✗ Branch 13431 → 13432 not taken.
✗ Branch 13431 → 19289 not taken.
 3 XmmReg r1, r2;
2666
1/4
✓ Branch 5187 → 5188 taken 3 times.
✗ Branch 5187 → 11044 not taken.
✗ Branch 13432 → 13433 not taken.
✗ Branch 13432 → 19289 not taken.
 3 Reg a;
2667
2/8
✓ Branch 5189 → 5190 taken 3 times.
✗ Branch 5189 → 11016 not taken.
✓ Branch 5190 → 5191 taken 3 times.
✗ Branch 5190 → 11016 not taken.
✗ Branch 13434 → 13435 not taken.
✗ Branch 13434 → 19261 not taken.
✗ Branch 13435 → 13436 not taken.
✗ Branch 13435 → 19261 not taken.
 3 addps(t1.first, CPTR(elfloat_half)); // rounder for truncate! no banker's rounding
2668
1/4
✓ Branch 5191 → 5192 taken 3 times.
✗ Branch 5191 → 11044 not taken.
✗ Branch 13436 → 13437 not taken.
✗ Branch 13436 → 19289 not taken.
 3 maxps(t1.first, zero);
2669
2/8
✓ Branch 5193 → 5194 taken 3 times.
✗ Branch 5193 → 11018 not taken.
✓ Branch 5194 → 5195 taken 3 times.
✗ Branch 5194 → 11018 not taken.
✗ Branch 13438 → 13439 not taken.
✗ Branch 13438 → 19263 not taken.
✗ Branch 13439 → 13440 not taken.
✗ Branch 13439 → 19263 not taken.
 3 addps(t1.second, CPTR(elfloat_half)); // rounder for truncate! no banker's rounding
2670
1/4
✓ Branch 5195 → 5196 taken 3 times.
✗ Branch 5195 → 11044 not taken.
✗ Branch 13440 → 13441 not taken.
✗ Branch 13440 → 19289 not taken.
 3 maxps(t1.second, zero);
2671
1/10
✗ Branch 5196 → 5197 not taken.
✗ Branch 5196 → 5204 not taken.
✗ Branch 5196 → 5211 not taken.
✓ Branch 5196 → 5218 taken 3 times.
✗ Branch 5196 → 5225 not taken.
✗ Branch 13441 → 13442 not taken.
✗ Branch 13441 → 13449 not taken.
✗ Branch 13441 → 13456 not taken.
✗ Branch 13441 → 13463 not taken.
✗ Branch 13441 → 13470 not taken.
 3 switch (iter.op) {
2672 ✗ case opStore10:
2673 ✗ minps(t1.first, CPTR(elstore10));
2674 ✗ minps(t1.second, CPTR(elstore10));
2675 ✗ break;
2676 ✗ case opStore12:
2677 ✗ minps(t1.first, CPTR(elstore12));
2678 ✗ minps(t1.second, CPTR(elstore12));
2679 ✗ break;
2680 ✗ case opStore14:
2681 ✗ minps(t1.first, CPTR(elstore14));
2682 ✗ minps(t1.second, CPTR(elstore14));
2683 ✗ break;
2684 3 case opStore16:
2685
2/8
✓ Branch 5219 → 5220 taken 3 times.
✗ Branch 5219 → 11032 not taken.
✓ Branch 5220 → 5221 taken 3 times.
✗ Branch 5220 → 11032 not taken.
✗ Branch 13464 → 13465 not taken.
✗ Branch 13464 → 19277 not taken.
✗ Branch 13465 → 13466 not taken.
✗ Branch 13465 → 19277 not taken.
 3 minps(t1.first, CPTR(elstore16));
2686
2/8
✓ Branch 5222 → 5223 taken 3 times.
✗ Branch 5222 → 11034 not taken.
✓ Branch 5223 → 5224 taken 3 times.
✗ Branch 5223 → 11034 not taken.
✗ Branch 13467 → 13468 not taken.
✗ Branch 13467 → 19279 not taken.
✗ Branch 13468 → 13469 not taken.
✗ Branch 13468 → 19279 not taken.
 3 minps(t1.second, CPTR(elstore16));
2687 3 break;
2688 }
2689
2/8
✓ Branch 5226 → 5227 taken 3 times.
✗ Branch 5226 → 11036 not taken.
✓ Branch 5227 → 5228 taken 3 times.
✗ Branch 5227 → 11036 not taken.
✗ Branch 13471 → 13472 not taken.
✗ Branch 13471 → 19281 not taken.
✗ Branch 13472 → 13473 not taken.
✗ Branch 13472 → 19281 not taken.
 3 mov(a, ptr[regptrs]);
2690
1/4
✓ Branch 5228 → 5229 taken 3 times.
✗ Branch 5228 → 11044 not taken.
✗ Branch 13473 → 13474 not taken.
✗ Branch 13473 → 19289 not taken.
 3 cvttps2dq(t1.first, t1.first); // no bankers rounding
2691
1/4
✓ Branch 5229 → 5230 taken 3 times.
✗ Branch 5229 → 11044 not taken.
✗ Branch 13474 → 13475 not taken.
✗ Branch 13474 → 19289 not taken.
 3 cvttps2dq(t1.second, t1.second);
2692 // new
2693
1/6
✗ Branch 5230 → 5231 not taken.
✓ Branch 5230 → 5233 taken 3 times.
✗ Branch 5230 → 5249 not taken.
✗ Branch 13475 → 13476 not taken.
✗ Branch 13475 → 13478 not taken.
✗ Branch 13475 → 13494 not taken.
 3 switch (iter.op) {
2694 ✗ case opStore10:
2695 case opStore12:
2696 case opStore14:
2697 ✗ packssdw(t1.first, t1.second); // _mm_packs_epi32: w7 w6 w5 w4 w3 w2 w1 w0
2698 ✗ break;
2699 3 case opStore16:
2700
1/4
✓ Branch 5233 → 5234 taken 3 times.
✗ Branch 5233 → 5235 not taken.
✗ Branch 13478 → 13479 not taken.
✗ Branch 13478 → 13480 not taken.
 3 if (cpuFlags & CPUF_SSE4_1) {
2701
1/4
✓ Branch 5234 → 5248 taken 3 times.
✗ Branch 5234 → 11044 not taken.
✗ Branch 13479 → 13493 not taken.
✗ Branch 13479 → 19289 not taken.
 3 packusdw(t1.first, t1.second); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
2702 }
2703 else {
2704 // old, sse2
2705 ✗ movdqa(r1, t1.first); // 00 w3 00 w2 00 w1 00 w0 -- min/max clamp ensures that high words are zero
2706 ✗ movdqa(r2, t1.second); // 00 w7 00 w6 00 w5 00 w4
2707 ✗ psrldq(t1.first, 6);
2708 ✗ psrldq(t1.second, 6);
2709 ✗ por(t1.first, r1);
2710 ✗ por(t1.second, r2);
2711 ✗ pshuflw(t1.first, t1.first, 0b11011000);
2712 ✗ pshuflw(t1.second, t1.second, 0b11011000);
2713 ✗ punpcklqdq(t1.first, t1.second);
2714 }
2715 3 break;
2716 }
2717
2/8
✓ Branch 5250 → 5251 taken 3 times.
✗ Branch 5250 → 11042 not taken.
✓ Branch 5251 → 5252 taken 3 times.
✗ Branch 5251 → 11042 not taken.
✗ Branch 13495 → 13496 not taken.
✗ Branch 13495 → 19287 not taken.
✗ Branch 13496 → 13497 not taken.
✗ Branch 13496 → 19287 not taken.
 3 movdqa(xmmword_ptr[a], t1.first);
2718 }
2719 6 }
2720
1/10
✗ Branch 616 → 617 not taken.
✗ Branch 616 → 627 not taken.
✓ Branch 5253 → 5254 taken 1 time.
✗ Branch 5253 → 5267 not taken.
✗ Branch 13498 → 13499 not taken.
✗ Branch 13498 → 13512 not taken.
✗ Branch 21274 → 21275 not taken.
✗ Branch 21274 → 21285 not taken.
✗ Branch 25730 → 25731 not taken.
✗ Branch 25730 → 25741 not taken.
 1 else if (iter.op == opStoreF32) {
2721 if (processSingle) {
2722 ✗ auto t1 = stack1.back();
2723 ✗ stack1.pop_back();
2724 ✗ Reg a;
2725 ✗ mov(a, ptr[regptrs]);
2726 ✗ movaps(xmmword_ptr[a], t1);
2727 }
2728 else {
2729 1 auto t1 = stack.back();
2730 1 stack.pop_back();
2731
1/4
✓ Branch 5256 → 5257 taken 1 time.
✗ Branch 5256 → 11051 not taken.
✗ Branch 13501 → 13502 not taken.
✗ Branch 13501 → 19296 not taken.
 1 Reg a;
2732
2/8
✓ Branch 5258 → 5259 taken 1 time.
✗ Branch 5258 → 11045 not taken.
✓ Branch 5259 → 5260 taken 1 time.
✗ Branch 5259 → 11045 not taken.
✗ Branch 13503 → 13504 not taken.
✗ Branch 13503 → 19290 not taken.
✗ Branch 13504 → 13505 not taken.
✗ Branch 13504 → 19290 not taken.
 1 mov(a, ptr[regptrs]);
2733
2/8
✓ Branch 5261 → 5262 taken 1 time.
✗ Branch 5261 → 11047 not taken.
✓ Branch 5262 → 5263 taken 1 time.
✗ Branch 5262 → 11047 not taken.
✗ Branch 13506 → 13507 not taken.
✗ Branch 13506 → 19292 not taken.
✗ Branch 13507 → 13508 not taken.
✗ Branch 13507 → 19292 not taken.
 1 movaps(xmmword_ptr[a], t1.first);
2734
2/8
✓ Branch 5264 → 5265 taken 1 time.
✗ Branch 5264 → 11049 not taken.
✓ Branch 5265 → 5266 taken 1 time.
✗ Branch 5265 → 11049 not taken.
✗ Branch 13509 → 13510 not taken.
✗ Branch 13509 → 19294 not taken.
✗ Branch 13510 → 13511 not taken.
✗ Branch 13510 → 19294 not taken.
 1 movaps(xmmword_ptr[a + 16], t1.second);
2735 }
2736 }
2737 ✗ else if (iter.op == opStoreF16) { // not supported in avs+
2738 if (processSingle) {
2739 ✗ auto t1 = stack1.back();
2740 ✗ stack1.pop_back();
2741 ✗ Reg a;
2742 ✗ mov(a, ptr[regptrs]);
2743 ✗ vcvtps2ph(qword_ptr[a], t1, 0);
2744 }
2745 else {
2746 ✗ auto t1 = stack.back();
2747 ✗ stack.pop_back();
2748 ✗ Reg a;
2749 ✗ mov(a, ptr[regptrs]);
2750 ✗ vcvtps2ph(qword_ptr[a], t1.first, 0);
2751 ✗ vcvtps2ph(qword_ptr[a + 8], t1.second, 0);
2752 }
2753 }
2754 ✗ else if (iter.op == opStoreVar || iter.op == opStoreVarAndDrop1) {
2755 if (processSingle) {
2756 ✗ auto t1 = stack1.back();
2757 // 16 bytes/variable
2758 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 16 * iter.e.ival;
2759 ✗ movaps(xmmword_ptr[regptrs + offset], t1);
2760 ✗ if (iter.op == opStoreVarAndDrop1)
2761 ✗ stack1.pop_back();
2762 }
2763 else {
2764 ✗ auto t1 = stack.back();
2765 // 32 byte/variable
2766 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 32 * iter.e.ival;
2767 ✗ movaps(xmmword_ptr[regptrs + offset], t1.first);
2768 ✗ movaps(xmmword_ptr[regptrs + offset + 16], t1.second);
2769 ✗ if (iter.op == opStoreVarAndDrop1)
2770 ✗ stack.pop_back();
2771 }
2772 ✗ }
2773 ✗ else if (iter.op == opAbs) {
2774 if (processSingle) {
2775 ✗ auto &t1 = stack1.back();
2776 ✗ andps(t1, CPTR(elabsmask));
2777 }
2778 else {
2779 ✗ auto &t1 = stack.back();
2780 ✗ andps(t1.first, CPTR(elabsmask));
2781 ✗ andps(t1.second, CPTR(elabsmask));
2782 }
2783 }
2784 ✗ else if (iter.op == opSgn) {
2785 // 1, 0, -1
2786 /*
2787 __m128 sgn(__m128 value) {
2788 const __m128 zero = _mm_set_ps1 (0.0f);
2789 __m128 p = _mm_and_ps(_mm_cmpgt_ps(value, zero), _mm_set_ps1(1.0f));
2790 __m128 n = _mm_and_ps(_mm_cmplt_ps(value, zero), _mm_set_ps1(-1.0f));
2791 return _mm_or_ps(p, n);
2792 }
2793 */
2794 if (processSingle) {
2795 ✗ auto &t1 = stack1.back();
2796 ✗ XmmReg r1, r2;
2797 ✗ xorps(r2, r2);
2798 ✗ movaps(r1, t1);
2799 ✗ movaps(t1, r2);
2800 ✗ cmpltps(t1, r1);
2801 ✗ cmpltps(r1, r2);
2802 ✗ andps(t1, CPTR(elfloat_one));
2803 ✗ andps(r1, CPTR(elfloat_minusone));
2804 ✗ orps(t1, r1);
2805 }
2806 else {
2807 ✗ auto &t1 = stack.back();
2808 ✗ XmmReg r2, r3, r4, r5;
2809 ✗ xorps(r2, r2);
2810 ✗ xorps(r3, r3);
2811 ✗ cmpltps(r3, t1.first);
2812 ✗ cmpltps(t1.first, r2);
2813 ✗ movaps(r4, t1.first);
2814 ✗ andnps(r4, r3);
2815 ✗ movaps(r3, CPTR(elfloat_one));
2816 ✗ xorps(r5, r5);
2817 ✗ cmpltps(r5, t1.second);
2818 ✗ cmpltps(t1.second, r2);
2819 ✗ movaps(r2, CPTR(elfloat_minusone));
2820 ✗ andps(t1.first, r2);
2821 ✗ andps(r2, t1.second);
2822 ✗ andnps(t1.second, r5);
2823 ✗ andps(r4, r3);
2824 ✗ orps(t1.first, r4);
2825 ✗ andps(t1.second, r3);
2826 ✗ orps(t1.second, r2);
2827 }
2828 }
2829 ✗ else if (iter.op == opNeg) {
2830 if (processSingle) {
2831 ✗ auto &t1 = stack1.back();
2832 ✗ cmpleps(t1, zero);
2833 ✗ andps(t1, CPTR(elfloat_one));
2834 }
2835 else {
2836 ✗ auto &t1 = stack.back();
2837 ✗ cmpleps(t1.first, zero);
2838 ✗ cmpleps(t1.second, zero);
2839 ✗ andps(t1.first, CPTR(elfloat_one));
2840 ✗ andps(t1.second, CPTR(elfloat_one));
2841 }
2842 }
2843 ✗ else if (iter.op == opNegSign) {
2844 if (processSingle) {
2845 ✗ auto& t1 = stack1.back();
2846 ✗ xorps(t1, CPTR(elsignmask));
2847 }
2848 else {
2849 ✗ auto& t1 = stack.back();
2850 ✗ xorps(t1.first, CPTR(elsignmask));
2851 ✗ xorps(t1.second, CPTR(elsignmask));
2852 }
2853 }
2854 ✗ else if (iter.op == opAnd) {
2855 if (processSingle) {
2856 ✗ LogicOp_Single(andps)
2857 }
2858 else {
2859 ✗ LogicOp(andps)
2860 }
2861 }
2862 ✗ else if (iter.op == opOr) {
2863 if (processSingle) {
2864 ✗ LogicOp_Single(orps)
2865 }
2866 else {
2867 ✗ LogicOp(orps)
2868 }
2869 }
2870 ✗ else if (iter.op == opXor) {
2871 if (processSingle) {
2872 ✗ LogicOp_Single(xorps)
2873 }
2874 else {
2875 ✗ LogicOp(xorps)
2876 }
2877 }
2878 ✗ else if (iter.op == opGt) { // a > b (gt) -> b < (lt) a
2879 if (processSingle) {
2880 ✗ CmpOp_Single(cmpltps)
2881 }
2882 else {
2883 ✗ CmpOp(cmpltps)
2884 }
2885 }
2886 ✗ else if (iter.op == opLt) { // a < b (lt) -> b > (gt,nle) a
2887 if (processSingle) {
2888 ✗ CmpOp_Single(cmpnleps)
2889 }
2890 else {
2891 ✗ CmpOp(cmpnleps)
2892 }
2893 }
2894 ✗ else if (iter.op == opEq) { // a == b -> b == a
2895 if (processSingle) {
2896 ✗ CmpOp_Single(cmpeqps)
2897 }
2898 else {
2899 ✗ CmpOp(cmpeqps)
2900 }
2901 }
2902 ✗ else if (iter.op == opNotEq) { // a != b
2903 if (processSingle) {
2904 ✗ CmpOp_Single(cmpneqps)
2905 }
2906 else {
2907 ✗ CmpOp(cmpneqps)
2908 }
2909 }
2910 ✗ else if (iter.op == opLE) { // a <= b -> b >= (ge,nlt) a
2911 if (processSingle) {
2912 ✗ CmpOp_Single(cmpnltps)
2913 }
2914 else {
2915 ✗ CmpOp(cmpnltps)
2916 }
2917 }
2918 ✗ else if (iter.op == opGE) { // a >= b -> b <= (le) a
2919 if (processSingle) {
2920 ✗ CmpOp_Single(cmpleps)
2921 }
2922 else {
2923 ✗ CmpOp(cmpleps)
2924 }
2925 }
2926 ✗ else if (iter.op == opTernary) {
2927 if (processSingle) {
2928 ✗ auto t1 = stack1.back();
2929 ✗ stack1.pop_back();
2930 ✗ auto t2 = stack1.back();
2931 ✗ stack1.pop_back();
2932 ✗ auto t3 = stack1.back();
2933 ✗ stack1.pop_back();
2934 ✗ XmmReg r1;
2935 ✗ xorps(r1, r1);
2936 ✗ cmpltps(r1, t3);
2937 ✗ andps(t2, r1);
2938 ✗ andnps(r1, t1);
2939 ✗ orps(r1, t2);
2940 ✗ stack1.push_back(r1);
2941 }
2942 else {
2943 ✗ auto t1 = stack.back();
2944 ✗ stack.pop_back();
2945 ✗ auto t2 = stack.back();
2946 ✗ stack.pop_back();
2947 ✗ auto t3 = stack.back();
2948 ✗ stack.pop_back();
2949 ✗ XmmReg r1, r2;
2950 ✗ xorps(r1, r1);
2951 ✗ xorps(r2, r2);
2952 ✗ cmpltps(r1, t3.first);
2953 ✗ cmpltps(r2, t3.second);
2954 ✗ andps(t2.first, r1);
2955 ✗ andps(t2.second, r2);
2956 ✗ andnps(r1, t1.first);
2957 ✗ andnps(r2, t1.second);
2958 ✗ orps(r1, t2.first);
2959 ✗ orps(r2, t2.second);
2960 ✗ stack.push_back(std::make_pair(r1, r2));
2961 }
2962 }
2963 ✗ else if (iter.op == opExp) {
2964 if (processSingle) {
2965 ✗ auto &t1 = stack1.back();
2966 ✗ EXP_PS(t1)
2967 }
2968 else {
2969 ✗ auto &t1 = stack.back();
2970 ✗ EXP_PS(t1.first)
2971 ✗ EXP_PS(t1.second)
2972 }
2973 }
2974 ✗ else if (iter.op == opLog) {
2975 if (processSingle) {
2976 ✗ auto &t1 = stack1.back();
2977 ✗ LOG_PS(t1)
2978 }
2979 else {
2980 ✗ auto &t1 = stack.back();
2981 ✗ LOG_PS(t1.first)
2982 ✗ LOG_PS(t1.second)
2983 }
2984 }
2985 ✗ else if (iter.op == opPow) {
2986 if (processSingle) {
2987 ✗ auto t1 = stack1.back();
2988 ✗ stack1.pop_back();
2989 ✗ auto &t2 = stack1.back();
2990 ✗ LOG_PS(t2)
2991 ✗ mulps(t2, t1);
2992 ✗ EXP_PS(t2)
2993 }
2994 else {
2995 ✗ auto t1 = stack.back();
2996 ✗ stack.pop_back();
2997 ✗ auto &t2 = stack.back();
2998 ✗ LOG_PS(t2.first)
2999 ✗ mulps(t2.first, t1.first);
3000 ✗ EXP_PS(t2.first)
3001 ✗ LOG_PS(t2.second)
3002 ✗ mulps(t2.second, t1.second);
3003 ✗ EXP_PS(t2.second)
3004 }
3005 }
3006 ✗ else if (iter.op == opSin) {
3007 if (processSingle) {
3008 ✗ auto& _t1 = stack1.back();
3009 ✗ SINCOS_PS(true, _t1, _t1)
3010 }
3011 else {
3012 ✗ auto& _t1 = stack.back();
3013 ✗ SINCOS_PS(true, _t1.first, _t1.first);
3014 ✗ SINCOS_PS(true, _t1.second, _t1.second);
3015 }
3016 }
3017 ✗ else if (iter.op == opCos) {
3018 if (processSingle) {
3019 ✗ auto& _t1 = stack1.back();
3020 ✗ SINCOS_PS(false, _t1, _t1)
3021 }
3022 else {
3023 ✗ auto& _t1 = stack.back();
3024 ✗ SINCOS_PS(false, _t1.first, _t1.first);
3025 ✗ SINCOS_PS(false, _t1.second, _t1.second);
3026 }
3027 }
3028 ✗ else if (iter.op == opTan) {
3029 if (processSingle) {
3030 ✗ auto& t1 = stack1.back();
3031 ✗ TAN_PS(t1)
3032 }
3033 else {
3034 ✗ auto& t1 = stack.back();
3035 ✗ TAN_PS(t1.first);
3036 ✗ TAN_PS(t1.second);
3037 }
3038 }
3039 ✗ else if (iter.op == opAtan2) {
3040 if (processSingle) {
3041 ✗ auto t1 = stack1.back();
3042 ✗ stack1.pop_back();
3043 ✗ auto& t2 = stack1.back();
3044 ✗ ATAN2_PS(t2, t1);
3045 }
3046 else {
3047 ✗ auto t1 = stack.back();
3048 ✗ stack.pop_back();
3049 ✗ auto& t2 = stack.back();
3050 ✗ ATAN2_PS(t2.first, t1.first);
3051 ✗ ATAN2_PS(t2.second, t1.second);
3052 }
3053 }
3054 ✗ else if (iter.op == opClip) {
3055 // clip(a, low, high) = min(max(a, low),high)
3056 if (processSingle) {
3057 ✗ auto t1 = stack1.back();
3058 ✗ stack1.pop_back();
3059 ✗ auto t2 = stack1.back();
3060 ✗ stack1.pop_back();
3061 ✗ auto &t3 = stack1.back();
3062 ✗ maxps(t3, t2);
3063 ✗ minps(t3, t1);
3064 }
3065 else {
3066 ✗ auto t1 = stack.back();
3067 ✗ stack.pop_back();
3068 ✗ auto t2 = stack.back();
3069 ✗ stack.pop_back();
3070 ✗ auto &t3 = stack.back();
3071 ✗ maxps(t3.first, t2.first);
3072 ✗ minps(t3.first, t1.first);
3073 ✗ maxps(t3.second, t2.second);
3074 ✗ minps(t3.second, t1.second);
3075 }
3076 }
3077 ✗ else if (iter.op == opRound || iter.op == opFloor || iter.op == opCeil || iter.op == opTrunc) {
3078 ✗ const int rounder_flag =
3079 ✗ (iter.op == opRound) ? (FROUND_TO_NEAREST_INT | FROUND_NO_EXC) :
3080 ✗ (iter.op == opFloor) ? (FROUND_TO_NEG_INF | FROUND_NO_EXC) :
3081 ✗ (iter.op == opCeil) ? (FROUND_TO_POS_INF | FROUND_NO_EXC) :
3082 (FROUND_TO_ZERO | FROUND_NO_EXC); // opTrunc
3083 if (processSingle) {
3084 ✗ auto& t1 = stack1.back();
3085 ✗ roundps(t1, t1, rounder_flag);
3086 }
3087 else {
3088 ✗ auto& t1 = stack.back();
3089 ✗ roundps(t1.first, t1.first, rounder_flag);
3090 ✗ roundps(t1.second, t1.second, rounder_flag);
3091 }
3092 }
3093
3094 }
3095 13 }
3096
3097 7 void main(Reg regptrs, Reg regoffs, Reg niter, Reg SpatialY)
3098 {
3099
1/2
✓ Branch 2 → 3 taken 7 times.
✗ Branch 2 → 29498 not taken.
 7 XmmReg zero;
3100
1/2
✓ Branch 3 → 4 taken 7 times.
✗ Branch 3 → 29498 not taken.
 7 pxor(zero, zero);
3101
1/2
✓ Branch 4 → 5 taken 7 times.
✗ Branch 4 → 29498 not taken.
 7 Reg constptr;
3102
2/4
✓ Branch 5 → 6 taken 7 times.
✗ Branch 5 → 29465 not taken.
✓ Branch 6 → 7 taken 7 times.
✗ Branch 6 → 29465 not taken.
 7 mov(constptr, (uintptr_t)logexpconst);
3103
3104
2/4
✓ Branch 9 → 10 taken 7 times.
✗ Branch 9 → 29468 not taken.
✓ Branch 10 → 11 taken 7 times.
✗ Branch 10 → 29466 not taken.
 14 L("wloop");
3105
2/4
✓ Branch 13 → 14 taken 7 times.
✗ Branch 13 → 29472 not taken.
✓ Branch 14 → 15 taken 7 times.
✗ Branch 14 → 29472 not taken.
 7 cmp(niter, 0);
3106
2/4
✓ Branch 17 → 18 taken 7 times.
✗ Branch 17 → 29475 not taken.
✓ Branch 18 → 19 taken 7 times.
✗ Branch 18 → 29473 not taken.
 14 je("wend");
3107 //sub(niter, 1);
3108
1/2
✓ Branch 21 → 22 taken 7 times.
✗ Branch 21 → 29498 not taken.
 7 dec(niter);
3109
3110 // process two sets, no partial input masking
3111
1/2
✗ Branch 22 → 23 not taken.
✓ Branch 22 → 4350 taken 7 times.
 7 if (singleMode)
3112 processingLoop<true, false>(regptrs, zero, constptr, SpatialY);
3113 else
3114 processingLoop<false, false>(regptrs, zero, constptr, SpatialY);
3115
3116 7 const int EXTRA = 2; // output pointer, xcounter
3117 if constexpr(sizeof(void *) == 8) {
3118 7 int numIter = (numInputs + EXTRA + 1) / 2;
3119
2/2
✓ Branch 12449 → 12436 taken 14 times.
✓ Branch 12449 → 12450 taken 7 times.
 21 for (int i = 0; i < numIter; i++) {
3120
2/4
✓ Branch 12436 → 12437 taken 14 times.
✗ Branch 12436 → 29485 not taken.
✓ Branch 12437 → 12438 taken 14 times.
✗ Branch 12437 → 29485 not taken.
 14 XmmReg r1, r2;
3121
2/4
✓ Branch 12439 → 12440 taken 14 times.
✗ Branch 12439 → 29479 not taken.
✓ Branch 12440 → 12441 taken 14 times.
✗ Branch 12440 → 29479 not taken.
 14 movdqu(r1, xmmword_ptr[regptrs + 16 * i]);
3122
2/4
✓ Branch 12442 → 12443 taken 14 times.
✗ Branch 12442 → 29481 not taken.
✓ Branch 12443 → 12444 taken 14 times.
✗ Branch 12443 → 29481 not taken.
 14 movdqu(r2, xmmword_ptr[regoffs + 16 * i]);
3123
1/2
✓ Branch 12444 → 12445 taken 14 times.
✗ Branch 12444 → 29485 not taken.
 14 paddq(r1, r2);
3124
2/4
✓ Branch 12446 → 12447 taken 14 times.
✗ Branch 12446 → 29483 not taken.
✓ Branch 12447 → 12448 taken 14 times.
✗ Branch 12447 → 29483 not taken.
 14 movdqu(xmmword_ptr[regptrs + 16 * i], r1);
3125 }
3126 } else {
3127 int numIter = (numInputs + EXTRA + 3) / 4;
3128 for (int i = 0; i < numIter; i++) {
3129 XmmReg r1, r2;
3130 movdqu(r1, xmmword_ptr[regptrs + 16 * i]);
3131 movdqu(r2, xmmword_ptr[regoffs + 16 * i]);
3132 paddd(r1, r2);
3133 movdqu(xmmword_ptr[regptrs + 16 * i], r1);
3134 }
3135 }
3136
3137
2/4
✓ Branch 12452 → 12453 taken 7 times.
✗ Branch 12452 → 29488 not taken.
✓ Branch 12453 → 12454 taken 7 times.
✗ Branch 12453 → 29486 not taken.
 14 jmp("wloop");
3138
2/4
✓ Branch 12458 → 12459 taken 7 times.
✗ Branch 12458 → 29494 not taken.
✓ Branch 12459 → 12460 taken 7 times.
✗ Branch 12459 → 29492 not taken.
 14 L("wend");
3139
3140
1/2
✗ Branch 12462 → 12463 not taken.
✓ Branch 12462 → 12464 taken 7 times.
 7 int nrestpixels = planewidth & (singleMode ? 3 : 7);
3141
2/2
✓ Branch 12465 → 12466 taken 3 times.
✓ Branch 12465 → 20680 taken 4 times.
 7 if (nrestpixels > 4) // dual process with masking
3142 processingLoop<false, true>(regptrs, zero, constptr, SpatialY);
3143
1/2
✗ Branch 20680 → 20681 not taken.
✓ Branch 20680 → 25008 taken 4 times.
 4 else if (nrestpixels == 4) // single process, no masking
3144 processingLoop<true, false>(regptrs, zero, constptr, SpatialY);
3145
2/2
✓ Branch 25008 → 25009 taken 3 times.
✓ Branch 25008 → 29464 taken 1 time.
 4 else if (nrestpixels > 0) // single process, masking
3146 processingLoop<true, true>(regptrs, zero, constptr, SpatialY);
3147 7 }
3148 };
3149
3150 // avx2 evaluator with two ymm registers
3151 struct ExprEvalAvx2 : public jitasm::function<void, ExprEvalAvx2, uint8_t *, const intptr_t *, intptr_t, intptr_t> {
3152
3153 std::vector<ExprOp> ops;
3154 int numInputs;
3155 int cpuFlags;
3156 int planewidth; // original, lut can overwrite
3157 int planeheight;
3158 bool singleMode;
3159
3160 18 ExprEvalAvx2(std::vector<ExprOp> &ops, int numInputs, int cpuFlags, int planewidth, int planeheight, bool singleMode) : ops(ops), numInputs(numInputs), cpuFlags(cpuFlags),
3161
1/2
✓ Branch 3 → 4 taken 6 times.
✗ Branch 3 → 5 not taken.
 6 planewidth(planewidth), planeheight(planeheight), singleMode(singleMode) {}
3162
3163 template<bool processSingle, bool maskUnused>
3164 AVS_FORCEINLINE void processingLoop(Reg &regptrs, YmmReg &zero, Reg &constptr, Reg &SpatialY)
3165 {
3166 12 std::list<std::pair<YmmReg, YmmReg>> stack;
3167 12 std::list<YmmReg> stack1;
3168
3169 // reason of masking: prevent loading 'junk', out of frame pixels, which can be NaN floats for example.
3170 // If processingLoop works in dual lane mode (!processSingle), masking occurs only for the high lane,
3171 // when there is no need for dual lanes (width mod 16 is <= 8 pixels), processSingle=true is used
3172 12 const bool maskIt = maskUnused && ((planewidth & 7) != 0);
3173 12 const int mask = ((1 << (planewidth & 7)) - 1);
3174
3175 // mask by zero when we have only 1-7 valid pixels in the lower (single-lane) or upper (dual-lane)
3176 // 1: 2-1 = 1 // 00000001
3177 // 2: 4-1 = 3 // 00000011
3178 // 7: 128-1 = 127 // 01111111
3179
3180
4/10
✗ Branch 1114 → 28 not taken.
✗ Branch 1114 → 1115 not taken.
✓ Branch 3418 → 1562 taken 26 times.
✓ Branch 3418 → 3419 taken 6 times.
✗ Branch 6148 → 4275 not taken.
✗ Branch 6148 → 6149 not taken.
✗ Branch 8066 → 6980 not taken.
✗ Branch 8066 → 8067 not taken.
✓ Branch 9618 → 8515 taken 26 times.
✓ Branch 9618 → 9619 taken 6 times.
 76 for (const auto &iter : ops) {
3181
4/10
✗ Branch 30 → 31 not taken.
✗ Branch 30 → 43 not taken.
✓ Branch 1564 → 1565 taken 2 times.
✓ Branch 1564 → 1583 taken 24 times.
✗ Branch 4277 → 4278 not taken.
✗ Branch 4277 → 4296 not taken.
✗ Branch 6982 → 6983 not taken.
✗ Branch 6982 → 6995 not taken.
✓ Branch 8517 → 8518 taken 2 times.
✓ Branch 8517 → 8530 taken 24 times.
 52 if (iter.op == opLoadSpatialX) {
3182 if (processSingle) {
3183
1/6
✗ Branch 31 → 32 not taken.
✗ Branch 31 → 1122 not taken.
✗ Branch 6983 → 6984 not taken.
✗ Branch 6983 → 8074 not taken.
✓ Branch 8518 → 8519 taken 2 times.
✗ Branch 8518 → 9626 not taken.
 2 YmmReg r1;
3184
1/6
✗ Branch 32 → 33 not taken.
✗ Branch 32 → 1122 not taken.
✗ Branch 6984 → 6985 not taken.
✗ Branch 6984 → 8074 not taken.
✓ Branch 8519 → 8520 taken 2 times.
✗ Branch 8519 → 9626 not taken.
 2 XmmReg r1x;
3185
2/12
✗ Branch 34 → 35 not taken.
✗ Branch 34 → 1118 not taken.
✗ Branch 35 → 36 not taken.
✗ Branch 35 → 1118 not taken.
✗ Branch 6986 → 6987 not taken.
✗ Branch 6986 → 8070 not taken.
✗ Branch 6987 → 6988 not taken.
✗ Branch 6987 → 8070 not taken.
✓ Branch 8521 → 8522 taken 2 times.
✗ Branch 8521 → 9622 not taken.
✓ Branch 8522 → 8523 taken 2 times.
✗ Branch 8522 → 9622 not taken.
 2 vmovd(r1x, dword_ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
3186
1/6
✗ Branch 36 → 37 not taken.
✗ Branch 36 → 1122 not taken.
✗ Branch 6988 → 6989 not taken.
✗ Branch 6988 → 8074 not taken.
✓ Branch 8523 → 8524 taken 2 times.
✗ Branch 8523 → 9626 not taken.
 2 vcvtdq2ps(r1x, r1x);
3187
1/6
✗ Branch 37 → 38 not taken.
✗ Branch 37 → 1122 not taken.
✗ Branch 6989 → 6990 not taken.
✗ Branch 6989 → 8074 not taken.
✓ Branch 8524 → 8525 taken 2 times.
✗ Branch 8524 → 9626 not taken.
 2 vbroadcastss(r1, r1x);
3188
2/12
✗ Branch 39 → 40 not taken.
✗ Branch 39 → 1120 not taken.
✗ Branch 40 → 41 not taken.
✗ Branch 40 → 1120 not taken.
✗ Branch 6991 → 6992 not taken.
✗ Branch 6991 → 8072 not taken.
✗ Branch 6992 → 6993 not taken.
✗ Branch 6992 → 8072 not taken.
✓ Branch 8526 → 8527 taken 2 times.
✗ Branch 8526 → 9624 not taken.
✓ Branch 8527 → 8528 taken 2 times.
✗ Branch 8527 → 9624 not taken.
 2 vaddps(r1, r1, CPTR_AVX(spatialX));
3189
1/6
✗ Branch 41 → 42 not taken.
✗ Branch 41 → 1122 not taken.
✗ Branch 6993 → 6994 not taken.
✗ Branch 6993 → 8074 not taken.
✓ Branch 8528 → 8529 taken 2 times.
✗ Branch 8528 → 9626 not taken.
 2 stack1.push_back(r1);
3190 }
3191 else {
3192
2/8
✓ Branch 1565 → 1566 taken 2 times.
✗ Branch 1565 → 3429 not taken.
✓ Branch 1566 → 1567 taken 2 times.
✗ Branch 1566 → 3429 not taken.
✗ Branch 4278 → 4279 not taken.
✗ Branch 4278 → 6159 not taken.
✗ Branch 4279 → 4280 not taken.
✗ Branch 4279 → 6159 not taken.
 2 YmmReg r1, r2;
3193
1/4
✓ Branch 1567 → 1568 taken 2 times.
✗ Branch 1567 → 3429 not taken.
✗ Branch 4280 → 4281 not taken.
✗ Branch 4280 → 6159 not taken.
 2 XmmReg r1x;
3194
2/8
✓ Branch 1569 → 1570 taken 2 times.
✗ Branch 1569 → 3422 not taken.
✓ Branch 1570 → 1571 taken 2 times.
✗ Branch 1570 → 3422 not taken.
✗ Branch 4282 → 4283 not taken.
✗ Branch 4282 → 6152 not taken.
✗ Branch 4283 → 4284 not taken.
✗ Branch 4283 → 6152 not taken.
 2 vmovd(r1x, dword_ptr[regptrs + sizeof(void *) * (RWPTR_START_OF_XCOUNTER)]);
3195
1/4
✓ Branch 1571 → 1572 taken 2 times.
✗ Branch 1571 → 3429 not taken.
✗ Branch 4284 → 4285 not taken.
✗ Branch 4284 → 6159 not taken.
 2 vcvtdq2ps(r1x, r1x);
3196
1/4
✓ Branch 1572 → 1573 taken 2 times.
✗ Branch 1572 → 3429 not taken.
✗ Branch 4285 → 4286 not taken.
✗ Branch 4285 → 6159 not taken.
 2 vbroadcastss(r1, r1x);
3197
1/4
✓ Branch 1573 → 1574 taken 2 times.
✗ Branch 1573 → 3429 not taken.
✗ Branch 4286 → 4287 not taken.
✗ Branch 4286 → 6159 not taken.
 2 vmovaps(r2, r1);
3198
2/8
✓ Branch 1575 → 1576 taken 2 times.
✗ Branch 1575 → 3424 not taken.
✓ Branch 1576 → 1577 taken 2 times.
✗ Branch 1576 → 3424 not taken.
✗ Branch 4288 → 4289 not taken.
✗ Branch 4288 → 6154 not taken.
✗ Branch 4289 → 4290 not taken.
✗ Branch 4289 → 6154 not taken.
 2 vaddps(r1, r1, CPTR_AVX(spatialX));
3199
2/8
✓ Branch 1578 → 1579 taken 2 times.
✗ Branch 1578 → 3426 not taken.
✓ Branch 1579 → 1580 taken 2 times.
✗ Branch 1579 → 3426 not taken.
✗ Branch 4291 → 4292 not taken.
✗ Branch 4291 → 6156 not taken.
✗ Branch 4292 → 4293 not taken.
✗ Branch 4292 → 6156 not taken.
 2 vaddps(r2, r2, CPTR_AVX(spatialX2));
3200
2/8
✓ Branch 1580 → 1581 taken 2 times.
✗ Branch 1580 → 3428 not taken.
✓ Branch 1581 → 1582 taken 2 times.
✗ Branch 1581 → 3428 not taken.
✗ Branch 4293 → 4294 not taken.
✗ Branch 4293 → 6158 not taken.
✗ Branch 4294 → 4295 not taken.
✗ Branch 4294 → 6158 not taken.
 2 stack.push_back(std::make_pair(r1, r2));
3201 }
3202 }
3203
4/10
✗ Branch 43 → 44 not taken.
✗ Branch 43 → 51 not taken.
✓ Branch 1583 → 1584 taken 2 times.
✓ Branch 1583 → 1594 taken 22 times.
✗ Branch 4296 → 4297 not taken.
✗ Branch 4296 → 4307 not taken.
✗ Branch 6995 → 6996 not taken.
✗ Branch 6995 → 7003 not taken.
✓ Branch 8530 → 8531 taken 2 times.
✓ Branch 8530 → 8538 taken 22 times.
 48 else if (iter.op == opLoadSpatialY) {
3204 if (processSingle) {
3205
1/6
✗ Branch 44 → 45 not taken.
✗ Branch 44 → 1123 not taken.
✗ Branch 6996 → 6997 not taken.
✗ Branch 6996 → 8075 not taken.
✓ Branch 8531 → 8532 taken 2 times.
✗ Branch 8531 → 9627 not taken.
 2 YmmReg r1;
3206
1/6
✗ Branch 45 → 46 not taken.
✗ Branch 45 → 1123 not taken.
✗ Branch 6997 → 6998 not taken.
✗ Branch 6997 → 8075 not taken.
✓ Branch 8532 → 8533 taken 2 times.
✗ Branch 8532 → 9627 not taken.
 2 XmmReg r1x;
3207 #ifdef JITASM64
3208
1/6
✗ Branch 46 → 47 not taken.
✗ Branch 46 → 1123 not taken.
✗ Branch 6998 → 6999 not taken.
✗ Branch 6998 → 8075 not taken.
✓ Branch 8533 → 8534 taken 2 times.
✗ Branch 8533 → 9627 not taken.
 2 vmovq(r1x, SpatialY);
3209 #else
3210 vmovd(r1x, SpatialY);
3211 #endif
3212
1/6
✗ Branch 47 → 48 not taken.
✗ Branch 47 → 1123 not taken.
✗ Branch 6999 → 7000 not taken.
✗ Branch 6999 → 8075 not taken.
✓ Branch 8534 → 8535 taken 2 times.
✗ Branch 8534 → 9627 not taken.
 2 vcvtdq2ps(r1x, r1x);
3213
1/6
✗ Branch 48 → 49 not taken.
✗ Branch 48 → 1123 not taken.
✗ Branch 7000 → 7001 not taken.
✗ Branch 7000 → 8075 not taken.
✓ Branch 8535 → 8536 taken 2 times.
✗ Branch 8535 → 9627 not taken.
 2 vbroadcastss(r1, r1x);
3214
1/6
✗ Branch 49 → 50 not taken.
✗ Branch 49 → 1123 not taken.
✗ Branch 7001 → 7002 not taken.
✗ Branch 7001 → 8075 not taken.
✓ Branch 8536 → 8537 taken 2 times.
✗ Branch 8536 → 9627 not taken.
 2 stack1.push_back(r1);
3215 }
3216 else {
3217
2/8
✓ Branch 1584 → 1585 taken 2 times.
✗ Branch 1584 → 3431 not taken.
✓ Branch 1585 → 1586 taken 2 times.
✗ Branch 1585 → 3431 not taken.
✗ Branch 4297 → 4298 not taken.
✗ Branch 4297 → 6161 not taken.
✗ Branch 4298 → 4299 not taken.
✗ Branch 4298 → 6161 not taken.
 2 YmmReg r1, r2;
3218
1/4
✓ Branch 1586 → 1587 taken 2 times.
✗ Branch 1586 → 3431 not taken.
✗ Branch 4299 → 4300 not taken.
✗ Branch 4299 → 6161 not taken.
 2 XmmReg r1x;
3219 #ifdef JITASM64
3220
1/4
✓ Branch 1587 → 1588 taken 2 times.
✗ Branch 1587 → 3431 not taken.
✗ Branch 4300 → 4301 not taken.
✗ Branch 4300 → 6161 not taken.
 2 vmovq(r1x, SpatialY);
3221 #else
3222 vmovd(r1x, SpatialY);
3223 #endif
3224
1/4
✓ Branch 1588 → 1589 taken 2 times.
✗ Branch 1588 → 3431 not taken.
✗ Branch 4301 → 4302 not taken.
✗ Branch 4301 → 6161 not taken.
 2 vcvtdq2ps(r1x, r1x);
3225
1/4
✓ Branch 1589 → 1590 taken 2 times.
✗ Branch 1589 → 3431 not taken.
✗ Branch 4302 → 4303 not taken.
✗ Branch 4302 → 6161 not taken.
 2 vbroadcastss(r1, r1x);
3226
1/4
✓ Branch 1590 → 1591 taken 2 times.
✗ Branch 1590 → 3431 not taken.
✗ Branch 4303 → 4304 not taken.
✗ Branch 4303 → 6161 not taken.
 2 vmovaps(r2, r1);
3227
2/8
✓ Branch 1591 → 1592 taken 2 times.
✗ Branch 1591 → 3430 not taken.
✓ Branch 1592 → 1593 taken 2 times.
✗ Branch 1592 → 3430 not taken.
✗ Branch 4304 → 4305 not taken.
✗ Branch 4304 → 6160 not taken.
✗ Branch 4305 → 4306 not taken.
✗ Branch 4305 → 6160 not taken.
 2 stack.push_back(std::make_pair(r1, r2));
3228 }
3229 }
3230
2/10
✗ Branch 51 → 52 not taken.
✗ Branch 51 → 60 not taken.
✗ Branch 1594 → 1595 not taken.
✓ Branch 1594 → 1606 taken 22 times.
✗ Branch 4307 → 4308 not taken.
✗ Branch 4307 → 4319 not taken.
✗ Branch 7003 → 7004 not taken.
✗ Branch 7003 → 7012 not taken.
✗ Branch 8538 → 8539 not taken.
✓ Branch 8538 → 8547 taken 22 times.
 44 else if (iter.op == opLoadInternalVar) {
3231 if (processSingle) {
3232 ✗ YmmReg r1;
3233 ✗ XmmReg r1x;
3234 ✗ vmovd(r1x, dword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INTERNAL_VARIABLES)]);
3235 ✗ vbroadcastss(r1, r1x);
3236 ✗ stack1.push_back(r1);
3237 }
3238 else {
3239 ✗ YmmReg r1, r2;
3240 ✗ XmmReg r1x;
3241 ✗ vmovd(r1x, dword_ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INTERNAL_VARIABLES)]);
3242 ✗ vbroadcastss(r1, r1x);
3243 ✗ vmovaps(r2, r1);
3244 ✗ stack.push_back(std::make_pair(r1, r2));
3245 }
3246 }
3247
2/10
✗ Branch 60 → 61 not taken.
✗ Branch 60 → 69 not taken.
✗ Branch 1606 → 1607 not taken.
✓ Branch 1606 → 1618 taken 22 times.
✗ Branch 4319 → 4320 not taken.
✗ Branch 4319 → 4331 not taken.
✗ Branch 7012 → 7013 not taken.
✗ Branch 7012 → 7021 not taken.
✗ Branch 8547 → 8548 not taken.
✓ Branch 8547 → 8556 taken 22 times.
 44 else if (iter.op == opLoadFramePropVar) {
3248 if (processSingle) {
3249 ✗ YmmReg r1;
3250 ✗ XmmReg r1x;
3251 ✗ vmovd(r1x, dword_ptr[regptrs + sizeof(void*) * (iter.e.ival + RWPTR_START_OF_INTERNAL_FRAMEPROP_VARIABLES)]);
3252 ✗ vbroadcastss(r1, r1x);
3253 ✗ stack1.push_back(r1);
3254 }
3255 else {
3256 ✗ YmmReg r1, r2;
3257 ✗ XmmReg r1x;
3258 ✗ vmovd(r1x, dword_ptr[regptrs + sizeof(void*) * (iter.e.ival + RWPTR_START_OF_INTERNAL_FRAMEPROP_VARIABLES)]);
3259 ✗ vbroadcastss(r1, r1x);
3260 ✗ vmovaps(r2, r1);
3261 ✗ stack.push_back(std::make_pair(r1, r2));
3262 }
3263 }
3264
4/10
✗ Branch 69 → 70 not taken.
✗ Branch 69 → 83 not taken.
✓ Branch 1618 → 1619 taken 3 times.
✓ Branch 1618 → 1641 taken 19 times.
✗ Branch 4331 → 4332 not taken.
✗ Branch 4331 → 4358 not taken.
✗ Branch 7021 → 7022 not taken.
✗ Branch 7021 → 7035 not taken.
✓ Branch 8556 → 8557 taken 3 times.
✓ Branch 8556 → 8574 taken 19 times.
 44 else if (iter.op == opLoadSrc8) {
3265 if (processSingle) {
3266
1/6
✗ Branch 70 → 71 not taken.
✗ Branch 70 → 1134 not taken.
✗ Branch 7022 → 7023 not taken.
✗ Branch 7022 → 8086 not taken.
✓ Branch 8557 → 8558 taken 3 times.
✗ Branch 8557 → 9639 not taken.
 3 XmmReg r1x;
3267
1/6
✗ Branch 71 → 72 not taken.
✗ Branch 71 → 1134 not taken.
✗ Branch 7023 → 7024 not taken.
✗ Branch 7023 → 8086 not taken.
✓ Branch 8558 → 8559 taken 3 times.
✗ Branch 8558 → 9639 not taken.
 3 YmmReg r1;
3268
1/6
✗ Branch 72 → 73 not taken.
✗ Branch 72 → 1134 not taken.
✗ Branch 7024 → 7025 not taken.
✗ Branch 7024 → 8086 not taken.
✓ Branch 8559 → 8560 taken 3 times.
✗ Branch 8559 → 9639 not taken.
 3 Reg a;
3269
2/12
✗ Branch 74 → 75 not taken.
✗ Branch 74 → 1130 not taken.
✗ Branch 75 → 76 not taken.
✗ Branch 75 → 1130 not taken.
✗ Branch 7026 → 7027 not taken.
✗ Branch 7026 → 8082 not taken.
✗ Branch 7027 → 7028 not taken.
✗ Branch 7027 → 8082 not taken.
✓ Branch 8561 → 8562 taken 3 times.
✗ Branch 8561 → 9634 not taken.
✓ Branch 8562 → 8563 taken 3 times.
✗ Branch 8562 → 9634 not taken.
 3 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3270 // 8 bytes, 8 pixels * uint8_t
3271
2/12
✗ Branch 77 → 78 not taken.
✗ Branch 77 → 1132 not taken.
✗ Branch 78 → 79 not taken.
✗ Branch 78 → 1132 not taken.
✗ Branch 7029 → 7030 not taken.
✗ Branch 7029 → 8084 not taken.
✗ Branch 7030 → 7031 not taken.
✗ Branch 7030 → 8084 not taken.
✓ Branch 8564 → 8565 taken 3 times.
✗ Branch 8564 → 9636 not taken.
✓ Branch 8565 → 8566 taken 3 times.
✗ Branch 8565 → 9636 not taken.
 3 vmovq(r1x, mmword_ptr[a]);
3272 // 8->32 bits like _mm256_cvtepu8_epi32
3273
1/6
✗ Branch 79 → 80 not taken.
✗ Branch 79 → 1134 not taken.
✗ Branch 7031 → 7032 not taken.
✗ Branch 7031 → 8086 not taken.
✓ Branch 8566 → 8567 taken 3 times.
✗ Branch 8566 → 9639 not taken.
 3 vpmovzxbd(r1, r1x);
3274 // int -> float
3275
1/6
✗ Branch 80 → 81 not taken.
✗ Branch 80 → 1134 not taken.
✗ Branch 7032 → 7033 not taken.
✗ Branch 7032 → 8086 not taken.
✓ Branch 8567 → 8568 taken 3 times.
✗ Branch 8567 → 9639 not taken.
 3 vcvtdq2ps(r1, r1);
3276
1/2
✓ Branch 8568 → 8569 taken 3 times.
✗ Branch 8568 → 8572 not taken.
 3 if (maskIt)
3277
2/4
✓ Branch 8569 → 8570 taken 3 times.
✗ Branch 8569 → 9638 not taken.
✓ Branch 8570 → 8571 taken 3 times.
✗ Branch 8570 → 9638 not taken.
 3 vblendps(r1, zero, r1, mask);
3278
1/6
✗ Branch 81 → 82 not taken.
✗ Branch 81 → 1134 not taken.
✗ Branch 7033 → 7034 not taken.
✗ Branch 7033 → 8086 not taken.
✓ Branch 8572 → 8573 taken 3 times.
✗ Branch 8572 → 9639 not taken.
 3 stack1.push_back(r1);
3279 }
3280 else {
3281
1/4
✓ Branch 1619 → 1620 taken 3 times.
✗ Branch 1619 → 3447 not taken.
✗ Branch 4332 → 4333 not taken.
✗ Branch 4332 → 6178 not taken.
 3 XmmReg r1x;
3282
2/8
✓ Branch 1620 → 1621 taken 3 times.
✗ Branch 1620 → 3447 not taken.
✓ Branch 1621 → 1622 taken 3 times.
✗ Branch 1621 → 3447 not taken.
✗ Branch 4333 → 4334 not taken.
✗ Branch 4333 → 6178 not taken.
✗ Branch 4334 → 4335 not taken.
✗ Branch 4334 → 6178 not taken.
 3 YmmReg r1, r2;
3283
1/4
✓ Branch 1622 → 1623 taken 3 times.
✗ Branch 1622 → 3447 not taken.
✗ Branch 4335 → 4336 not taken.
✗ Branch 4335 → 6178 not taken.
 3 Reg a;
3284
2/8
✓ Branch 1624 → 1625 taken 3 times.
✗ Branch 1624 → 3440 not taken.
✓ Branch 1625 → 1626 taken 3 times.
✗ Branch 1625 → 3440 not taken.
✗ Branch 4337 → 4338 not taken.
✗ Branch 4337 → 6170 not taken.
✗ Branch 4338 → 4339 not taken.
✗ Branch 4338 → 6170 not taken.
 3 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3285 // 16 bytes, 16 pixels * uint8_t
3286
2/8
✓ Branch 1627 → 1628 taken 3 times.
✗ Branch 1627 → 3442 not taken.
✓ Branch 1628 → 1629 taken 3 times.
✗ Branch 1628 → 3442 not taken.
✗ Branch 4340 → 4341 not taken.
✗ Branch 4340 → 6172 not taken.
✗ Branch 4341 → 4342 not taken.
✗ Branch 4341 → 6172 not taken.
 3 vmovdqa(r1x, xmmword_ptr[a]);
3287 // 8->16 bits like _mm256_cvtepu8_epi16
3288
1/4
✓ Branch 1629 → 1630 taken 3 times.
✗ Branch 1629 → 3447 not taken.
✗ Branch 4342 → 4343 not taken.
✗ Branch 4342 → 6178 not taken.
 3 vpmovzxbw(r1, r1x);
3289 // 16->32 bit like _mm256_cvtepu16_epi32
3290
2/8
✓ Branch 1630 → 1631 taken 3 times.
✗ Branch 1630 → 3444 not taken.
✓ Branch 1631 → 1632 taken 3 times.
✗ Branch 1631 → 3444 not taken.
✗ Branch 4343 → 4344 not taken.
✗ Branch 4343 → 6174 not taken.
✗ Branch 4344 → 4345 not taken.
✗ Branch 4344 → 6174 not taken.
 3 vextracti128(r1x, r1, 1); // upper 128
3291
1/4
✓ Branch 1632 → 1633 taken 3 times.
✗ Branch 1632 → 3447 not taken.
✗ Branch 4345 → 4346 not taken.
✗ Branch 4345 → 6178 not taken.
 3 vpmovzxwd(r2, r1x);
3292
2/8
✓ Branch 1633 → 1634 taken 3 times.
✗ Branch 1633 → 3445 not taken.
✓ Branch 1634 → 1635 taken 3 times.
✗ Branch 1634 → 3445 not taken.
✗ Branch 4346 → 4347 not taken.
✗ Branch 4346 → 6175 not taken.
✗ Branch 4347 → 4348 not taken.
✗ Branch 4347 → 6175 not taken.
 3 vextracti128(r1x, r1, 0); // lower 128
3293
1/4
✓ Branch 1635 → 1636 taken 3 times.
✗ Branch 1635 → 3447 not taken.
✗ Branch 4348 → 4349 not taken.
✗ Branch 4348 → 6178 not taken.
 3 vpmovzxwd(r1, r1x);
3294 // int -> float
3295
1/4
✓ Branch 1636 → 1637 taken 3 times.
✗ Branch 1636 → 3447 not taken.
✗ Branch 4349 → 4350 not taken.
✗ Branch 4349 → 6178 not taken.
 3 vcvtdq2ps(r1, r1);
3296
1/4
✓ Branch 1637 → 1638 taken 3 times.
✗ Branch 1637 → 3447 not taken.
✗ Branch 4350 → 4351 not taken.
✗ Branch 4350 → 6178 not taken.
 3 vcvtdq2ps(r2, r2);
3297 ✗ if (maskIt)
3298 ✗ vblendps(r2, zero, r2, mask);
3299
2/8
✓ Branch 1638 → 1639 taken 3 times.
✗ Branch 1638 → 3446 not taken.
✓ Branch 1639 → 1640 taken 3 times.
✗ Branch 1639 → 3446 not taken.
✗ Branch 4355 → 4356 not taken.
✗ Branch 4355 → 6177 not taken.
✗ Branch 4356 → 4357 not taken.
✗ Branch 4356 → 6177 not taken.
 3 stack.push_back(std::make_pair(r1, r2));
3300 }
3301 }
3302
4/10
✗ Branch 83 → 84 not taken.
✗ Branch 83 → 97 not taken.
✓ Branch 1641 → 1642 taken 3 times.
✓ Branch 1641 → 1663 taken 16 times.
✗ Branch 4358 → 4359 not taken.
✗ Branch 4358 → 4384 not taken.
✗ Branch 7035 → 7036 not taken.
✗ Branch 7035 → 7049 not taken.
✓ Branch 8574 → 8575 taken 3 times.
✓ Branch 8574 → 8592 taken 16 times.
 38 else if (iter.op == opLoadSrc16) {
3303 if (processSingle) {
3304
1/6
✗ Branch 84 → 85 not taken.
✗ Branch 84 → 1139 not taken.
✗ Branch 7036 → 7037 not taken.
✗ Branch 7036 → 8091 not taken.
✓ Branch 8575 → 8576 taken 3 times.
✗ Branch 8575 → 9645 not taken.
 3 XmmReg r1x;
3305
1/6
✗ Branch 85 → 86 not taken.
✗ Branch 85 → 1139 not taken.
✗ Branch 7037 → 7038 not taken.
✗ Branch 7037 → 8091 not taken.
✓ Branch 8576 → 8577 taken 3 times.
✗ Branch 8576 → 9645 not taken.
 3 YmmReg r1;
3306
1/6
✗ Branch 86 → 87 not taken.
✗ Branch 86 → 1139 not taken.
✗ Branch 7038 → 7039 not taken.
✗ Branch 7038 → 8091 not taken.
✓ Branch 8577 → 8578 taken 3 times.
✗ Branch 8577 → 9645 not taken.
 3 Reg a;
3307
2/12
✗ Branch 88 → 89 not taken.
✗ Branch 88 → 1135 not taken.
✗ Branch 89 → 90 not taken.
✗ Branch 89 → 1135 not taken.
✗ Branch 7040 → 7041 not taken.
✗ Branch 7040 → 8087 not taken.
✗ Branch 7041 → 7042 not taken.
✗ Branch 7041 → 8087 not taken.
✓ Branch 8579 → 8580 taken 3 times.
✗ Branch 8579 → 9640 not taken.
✓ Branch 8580 → 8581 taken 3 times.
✗ Branch 8580 → 9640 not taken.
 3 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3308 // 16 bytes, 8 pixels * uint16_t
3309
2/12
✗ Branch 91 → 92 not taken.
✗ Branch 91 → 1137 not taken.
✗ Branch 92 → 93 not taken.
✗ Branch 92 → 1137 not taken.
✗ Branch 7043 → 7044 not taken.
✗ Branch 7043 → 8089 not taken.
✗ Branch 7044 → 7045 not taken.
✗ Branch 7044 → 8089 not taken.
✓ Branch 8582 → 8583 taken 3 times.
✗ Branch 8582 → 9642 not taken.
✓ Branch 8583 → 8584 taken 3 times.
✗ Branch 8583 → 9642 not taken.
 3 vmovdqa(r1x, xmmword_ptr[a]);
3310 // 16->32 bit like _mm256_cvtepu16_epi32
3311
1/6
✗ Branch 93 → 94 not taken.
✗ Branch 93 → 1139 not taken.
✗ Branch 7045 → 7046 not taken.
✗ Branch 7045 → 8091 not taken.
✓ Branch 8584 → 8585 taken 3 times.
✗ Branch 8584 → 9645 not taken.
 3 vpmovzxwd(r1, r1x);
3312 // int -> float
3313
1/6
✗ Branch 94 → 95 not taken.
✗ Branch 94 → 1139 not taken.
✗ Branch 7046 → 7047 not taken.
✗ Branch 7046 → 8091 not taken.
✓ Branch 8585 → 8586 taken 3 times.
✗ Branch 8585 → 9645 not taken.
 3 vcvtdq2ps(r1, r1);
3314
1/2
✓ Branch 8586 → 8587 taken 3 times.
✗ Branch 8586 → 8590 not taken.
 3 if (maskIt)
3315
2/4
✓ Branch 8587 → 8588 taken 3 times.
✗ Branch 8587 → 9644 not taken.
✓ Branch 8588 → 8589 taken 3 times.
✗ Branch 8588 → 9644 not taken.
 3 vblendps(r1, zero, r1, mask);
3316
1/6
✗ Branch 95 → 96 not taken.
✗ Branch 95 → 1139 not taken.
✗ Branch 7047 → 7048 not taken.
✗ Branch 7047 → 8091 not taken.
✓ Branch 8590 → 8591 taken 3 times.
✗ Branch 8590 → 9645 not taken.
 3 stack1.push_back(r1);
3317 }
3318 else {
3319
1/4
✓ Branch 1642 → 1643 taken 3 times.
✗ Branch 1642 → 3455 not taken.
✗ Branch 4359 → 4360 not taken.
✗ Branch 4359 → 6187 not taken.
 3 XmmReg r1x;
3320
2/8
✓ Branch 1643 → 1644 taken 3 times.
✗ Branch 1643 → 3455 not taken.
✓ Branch 1644 → 1645 taken 3 times.
✗ Branch 1644 → 3455 not taken.
✗ Branch 4360 → 4361 not taken.
✗ Branch 4360 → 6187 not taken.
✗ Branch 4361 → 4362 not taken.
✗ Branch 4361 → 6187 not taken.
 3 YmmReg r1, r2;
3321
1/4
✓ Branch 1645 → 1646 taken 3 times.
✗ Branch 1645 → 3455 not taken.
✗ Branch 4362 → 4363 not taken.
✗ Branch 4362 → 6187 not taken.
 3 Reg a;
3322
2/8
✓ Branch 1647 → 1648 taken 3 times.
✗ Branch 1647 → 3448 not taken.
✓ Branch 1648 → 1649 taken 3 times.
✗ Branch 1648 → 3448 not taken.
✗ Branch 4364 → 4365 not taken.
✗ Branch 4364 → 6179 not taken.
✗ Branch 4365 → 4366 not taken.
✗ Branch 4365 → 6179 not taken.
 3 mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3323 // 32 bytes, 16 pixels * uint16_t
3324
2/8
✓ Branch 1650 → 1651 taken 3 times.
✗ Branch 1650 → 3450 not taken.
✓ Branch 1651 → 1652 taken 3 times.
✗ Branch 1651 → 3450 not taken.
✗ Branch 4367 → 4368 not taken.
✗ Branch 4367 → 6181 not taken.
✗ Branch 4368 → 4369 not taken.
✗ Branch 4368 → 6181 not taken.
 3 vmovdqa(r1, ymmword_ptr[a]);
3325 // 16->32 bit like _mm256_cvtepu16_epi32
3326
2/8
✓ Branch 1652 → 1653 taken 3 times.
✗ Branch 1652 → 3452 not taken.
✓ Branch 1653 → 1654 taken 3 times.
✗ Branch 1653 → 3452 not taken.
✗ Branch 4369 → 4370 not taken.
✗ Branch 4369 → 6183 not taken.
✗ Branch 4370 → 4371 not taken.
✗ Branch 4370 → 6183 not taken.
 3 vextracti128(r1x, r1, 1); // upper 128
3327
1/4
✓ Branch 1654 → 1655 taken 3 times.
✗ Branch 1654 → 3455 not taken.
✗ Branch 4371 → 4372 not taken.
✗ Branch 4371 → 6187 not taken.
 3 vpmovzxwd(r2, r1x);
3328
2/8
✓ Branch 1655 → 1656 taken 3 times.
✗ Branch 1655 → 3453 not taken.
✓ Branch 1656 → 1657 taken 3 times.
✗ Branch 1656 → 3453 not taken.
✗ Branch 4372 → 4373 not taken.
✗ Branch 4372 → 6184 not taken.
✗ Branch 4373 → 4374 not taken.
✗ Branch 4373 → 6184 not taken.
 3 vextracti128(r1x, r1, 0); // lower 128
3329
1/4
✓ Branch 1657 → 1658 taken 3 times.
✗ Branch 1657 → 3455 not taken.
✗ Branch 4374 → 4375 not taken.
✗ Branch 4374 → 6187 not taken.
 3 vpmovzxwd(r1, r1x);
3330 // int -> float
3331
1/4
✓ Branch 1658 → 1659 taken 3 times.
✗ Branch 1658 → 3455 not taken.
✗ Branch 4375 → 4376 not taken.
✗ Branch 4375 → 6187 not taken.
 3 vcvtdq2ps(r1, r1);
3332
1/4
✓ Branch 1659 → 1660 taken 3 times.
✗ Branch 1659 → 3455 not taken.
✗ Branch 4376 → 4377 not taken.
✗ Branch 4376 → 6187 not taken.
 3 vcvtdq2ps(r2, r2);
3333 ✗ if (maskIt)
3334 ✗ vblendps(r2, zero, r2, mask);
3335
2/8
✓ Branch 1660 → 1661 taken 3 times.
✗ Branch 1660 → 3454 not taken.
✓ Branch 1661 → 1662 taken 3 times.
✗ Branch 1661 → 3454 not taken.
✗ Branch 4381 → 4382 not taken.
✗ Branch 4381 → 6186 not taken.
✗ Branch 4382 → 4383 not taken.
✗ Branch 4382 → 6186 not taken.
 3 stack.push_back(std::make_pair(r1, r2));
3336 }
3337 }
3338
2/10
✗ Branch 97 → 98 not taken.
✗ Branch 97 → 109 not taken.
✗ Branch 1663 → 1664 not taken.
✓ Branch 1663 → 1680 taken 16 times.
✗ Branch 4384 → 4385 not taken.
✗ Branch 4384 → 4404 not taken.
✗ Branch 7049 → 7050 not taken.
✗ Branch 7049 → 7061 not taken.
✗ Branch 8592 → 8593 not taken.
✓ Branch 8592 → 8607 taken 16 times.
 32 else if (iter.op == opLoadSrcF32) {
3339 if (processSingle) {
3340 ✗ YmmReg r1;
3341 ✗ Reg a;
3342 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3343 // 32 bytes, 8 * float
3344 ✗ vmovdqa(r1, ymmword_ptr[a]);
3345 ✗ if (maskIt)
3346 ✗ vblendps(r1, zero, r1, mask);
3347 ✗ stack1.push_back(r1);
3348 }
3349 else {
3350 ✗ YmmReg r1, r2;
3351 ✗ Reg a;
3352 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3353 // 32 bytes, 8 * float
3354 ✗ vmovdqa(r1, ymmword_ptr[a]);
3355 ✗ vmovdqa(r2, ymmword_ptr[a + 32]); // needs 64 byte aligned data to prevent read past valid data!
3356 ✗ if (maskIt)
3357 ✗ vblendps(r2, zero, r2, mask);
3358 ✗ stack.push_back(std::make_pair(r1, r2));
3359 }
3360 }
3361
2/10
✗ Branch 109 → 110 not taken.
✗ Branch 109 → 121 not taken.
✗ Branch 1680 → 1681 not taken.
✓ Branch 1680 → 1697 taken 16 times.
✗ Branch 4404 → 4405 not taken.
✗ Branch 4404 → 4424 not taken.
✗ Branch 7061 → 7062 not taken.
✗ Branch 7061 → 7073 not taken.
✗ Branch 8607 → 8608 not taken.
✓ Branch 8607 → 8622 taken 16 times.
 32 else if (iter.op == opLoadSrcF16) { // not supported in avs+
3362 if (processSingle) {
3363 ✗ YmmReg r1;
3364 ✗ Reg a;
3365 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3366 ✗ vcvtph2ps(r1, xmmword_ptr[a]);
3367 ✗ if (maskIt)
3368 ✗ vblendps(r1, zero, r1, mask);
3369 ✗ stack1.push_back(r1);
3370 }
3371 else {
3372 ✗ YmmReg r1, r2;
3373 ✗ Reg a;
3374 ✗ mov(a, ptr[regptrs + sizeof(void *) * (iter.e.ival + RWPTR_START_OF_INPUTS)]);
3375 ✗ vcvtph2ps(r1, xmmword_ptr[a]);
3376 ✗ vcvtph2ps(r2, xmmword_ptr[a + 16]);
3377 ✗ if (maskIt)
3378 ✗ vblendps(r2, zero, r2, mask);
3379 ✗ stack.push_back(std::make_pair(r1, r2));
3380 }
3381 }
3382
2/10
✗ Branch 121 → 122 not taken.
✗ Branch 121 → 129 not taken.
✗ Branch 1697 → 1698 not taken.
✓ Branch 1697 → 1711 taken 16 times.
✗ Branch 4424 → 4425 not taken.
✗ Branch 4424 → 4441 not taken.
✗ Branch 7073 → 7074 not taken.
✗ Branch 7073 → 7081 not taken.
✗ Branch 8622 → 8623 not taken.
✓ Branch 8622 → 8633 taken 16 times.
 32 else if (iter.op == opLoadVar) {
3383 if (processSingle) {
3384 ✗ YmmReg r1;
3385 // 32 bytes/variable
3386 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 32 * iter.e.ival;
3387 // 32 bytes, 8 * float
3388 ✗ vmovdqa(r1, ymmword_ptr[regptrs + offset]);
3389 ✗ if (maskIt)
3390 ✗ vblendps(r1, zero, r1, mask);
3391 ✗ stack1.push_back(r1);
3392 }
3393 else {
3394 ✗ YmmReg r1, r2;
3395 // 64 bytes/variable
3396 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 64 * iter.e.ival;
3397 // 32 bytes, 8 * float
3398 ✗ vmovdqa(r1, ymmword_ptr[regptrs + offset]);
3399 ✗ vmovdqa(r2, ymmword_ptr[regptrs + offset + 32]); // needs 64 byte aligned data to prevent read past valid data!
3400 ✗ if (maskIt)
3401 ✗ vblendps(r2, zero, r2, mask);
3402 ✗ stack.push_back(std::make_pair(r1, r2));
3403 }
3404 }
3405
4/10
✗ Branch 129 → 130 not taken.
✗ Branch 129 → 139 not taken.
✓ Branch 1711 → 1712 taken 3 times.
✓ Branch 1711 → 1724 taken 13 times.
✗ Branch 4441 → 4442 not taken.
✗ Branch 4441 → 4454 not taken.
✗ Branch 7081 → 7082 not taken.
✗ Branch 7081 → 7091 not taken.
✓ Branch 8633 → 8634 taken 3 times.
✓ Branch 8633 → 8643 taken 13 times.
 32 else if (iter.op == opLoadConst) {
3406 if (processSingle) {
3407
1/6
✗ Branch 130 → 131 not taken.
✗ Branch 130 → 1154 not taken.
✗ Branch 7082 → 7083 not taken.
✗ Branch 7082 → 8106 not taken.
✓ Branch 8634 → 8635 taken 3 times.
✗ Branch 8634 → 9663 not taken.
 3 YmmReg r1;
3408
1/6
✗ Branch 131 → 132 not taken.
✗ Branch 131 → 1154 not taken.
✗ Branch 7083 → 7084 not taken.
✗ Branch 7083 → 8106 not taken.
✓ Branch 8635 → 8636 taken 3 times.
✗ Branch 8635 → 9663 not taken.
 3 Reg32 a;
3409
1/6
✗ Branch 132 → 133 not taken.
✗ Branch 132 → 1154 not taken.
✗ Branch 7084 → 7085 not taken.
✗ Branch 7084 → 8106 not taken.
✓ Branch 8636 → 8637 taken 3 times.
✗ Branch 8636 → 9663 not taken.
 3 XmmReg r1x;
3410
2/12
✗ Branch 133 → 134 not taken.
✗ Branch 133 → 1153 not taken.
✗ Branch 134 → 135 not taken.
✗ Branch 134 → 1153 not taken.
✗ Branch 7085 → 7086 not taken.
✗ Branch 7085 → 8105 not taken.
✗ Branch 7086 → 7087 not taken.
✗ Branch 7086 → 8105 not taken.
✓ Branch 8637 → 8638 taken 3 times.
✗ Branch 8637 → 9662 not taken.
✓ Branch 8638 → 8639 taken 3 times.
✗ Branch 8638 → 9662 not taken.
 3 mov(a, iter.e.ival);
3411
1/6
✗ Branch 135 → 136 not taken.
✗ Branch 135 → 1154 not taken.
✗ Branch 7087 → 7088 not taken.
✗ Branch 7087 → 8106 not taken.
✓ Branch 8639 → 8640 taken 3 times.
✗ Branch 8639 → 9663 not taken.
 3 vmovd(r1x, a);
3412
1/6
✗ Branch 136 → 137 not taken.
✗ Branch 136 → 1154 not taken.
✗ Branch 7088 → 7089 not taken.
✗ Branch 7088 → 8106 not taken.
✓ Branch 8640 → 8641 taken 3 times.
✗ Branch 8640 → 9663 not taken.
 3 vbroadcastss(r1, r1x);
3413
1/6
✗ Branch 137 → 138 not taken.
✗ Branch 137 → 1154 not taken.
✗ Branch 7089 → 7090 not taken.
✗ Branch 7089 → 8106 not taken.
✓ Branch 8641 → 8642 taken 3 times.
✗ Branch 8641 → 9663 not taken.
 3 stack1.push_back(r1);
3414 }
3415 else {
3416
2/8
✓ Branch 1712 → 1713 taken 3 times.
✗ Branch 1712 → 3481 not taken.
✓ Branch 1713 → 1714 taken 3 times.
✗ Branch 1713 → 3481 not taken.
✗ Branch 4442 → 4443 not taken.
✗ Branch 4442 → 6216 not taken.
✗ Branch 4443 → 4444 not taken.
✗ Branch 4443 → 6216 not taken.
 3 YmmReg r1, r2;
3417
1/4
✓ Branch 1714 → 1715 taken 3 times.
✗ Branch 1714 → 3481 not taken.
✗ Branch 4444 → 4445 not taken.
✗ Branch 4444 → 6216 not taken.
 3 Reg32 a;
3418
1/4
✓ Branch 1715 → 1716 taken 3 times.
✗ Branch 1715 → 3481 not taken.
✗ Branch 4445 → 4446 not taken.
✗ Branch 4445 → 6216 not taken.
 3 XmmReg r1x;
3419
2/8
✓ Branch 1716 → 1717 taken 3 times.
✗ Branch 1716 → 3479 not taken.
✓ Branch 1717 → 1718 taken 3 times.
✗ Branch 1717 → 3479 not taken.
✗ Branch 4446 → 4447 not taken.
✗ Branch 4446 → 6214 not taken.
✗ Branch 4447 → 4448 not taken.
✗ Branch 4447 → 6214 not taken.
 3 mov(a, iter.e.ival);
3420
1/4
✓ Branch 1718 → 1719 taken 3 times.
✗ Branch 1718 → 3481 not taken.
✗ Branch 4448 → 4449 not taken.
✗ Branch 4448 → 6216 not taken.
 3 vmovd(r1x, a);
3421
1/4
✓ Branch 1719 → 1720 taken 3 times.
✗ Branch 1719 → 3481 not taken.
✗ Branch 4449 → 4450 not taken.
✗ Branch 4449 → 6216 not taken.
 3 vbroadcastss(r1, r1x);
3422
1/4
✓ Branch 1720 → 1721 taken 3 times.
✗ Branch 1720 → 3481 not taken.
✗ Branch 4450 → 4451 not taken.
✗ Branch 4450 → 6216 not taken.
 3 vmovaps(r2, r1);
3423
2/8
✓ Branch 1721 → 1722 taken 3 times.
✗ Branch 1721 → 3480 not taken.
✓ Branch 1722 → 1723 taken 3 times.
✗ Branch 1722 → 3480 not taken.
✗ Branch 4451 → 4452 not taken.
✗ Branch 4451 → 6215 not taken.
✗ Branch 4452 → 4453 not taken.
✗ Branch 4452 → 6215 not taken.
 3 stack.push_back(std::make_pair(r1, r2));
3424 }
3425 }
3426
2/10
✗ Branch 139 → 140 not taken.
✗ Branch 139 → 152 not taken.
✗ Branch 1724 → 1725 not taken.
✓ Branch 1724 → 1741 taken 13 times.
✗ Branch 4454 → 4455 not taken.
✗ Branch 4454 → 4471 not taken.
✗ Branch 7091 → 7092 not taken.
✗ Branch 7091 → 7104 not taken.
✗ Branch 8643 → 8644 not taken.
✓ Branch 8643 → 8656 taken 13 times.
 26 else if (iter.op == opDup) {
3427 if (processSingle) {
3428 ✗ auto p = std::next(stack1.rbegin(), iter.e.ival);
3429 ✗ YmmReg r1;
3430 ✗ vmovaps(r1, *p);
3431 ✗ stack1.push_back(r1);
3432 }
3433 else {
3434 ✗ auto p = std::next(stack.rbegin(), iter.e.ival);
3435 ✗ YmmReg r1, r2;
3436 ✗ vmovaps(r1, p->first);
3437 ✗ vmovaps(r2, p->second);
3438 ✗ stack.push_back(std::make_pair(r1, r2));
3439 }
3440 }
3441
2/10
✗ Branch 152 → 153 not taken.
✗ Branch 152 → 164 not taken.
✗ Branch 1741 → 1742 not taken.
✓ Branch 1741 → 1753 taken 13 times.
✗ Branch 4471 → 4472 not taken.
✗ Branch 4471 → 4483 not taken.
✗ Branch 7104 → 7105 not taken.
✗ Branch 7104 → 7116 not taken.
✗ Branch 8656 → 8657 not taken.
✓ Branch 8656 → 8668 taken 13 times.
 26 else if (iter.op == opSwap) {
3442 if(processSingle)
3443 ✗ std::swap(stack1.back(), *std::next(stack1.rbegin(), iter.e.ival));
3444 else
3445 ✗ std::swap(stack.back(), *std::next(stack.rbegin(), iter.e.ival));
3446 }
3447
4/10
✗ Branch 164 → 165 not taken.
✗ Branch 164 → 170 not taken.
✓ Branch 1753 → 1754 taken 7 times.
✓ Branch 1753 → 1760 taken 6 times.
✗ Branch 4483 → 4484 not taken.
✗ Branch 4483 → 4490 not taken.
✗ Branch 7116 → 7117 not taken.
✗ Branch 7116 → 7122 not taken.
✓ Branch 8668 → 8669 taken 7 times.
✓ Branch 8668 → 8674 taken 6 times.
 26 else if (iter.op == opAdd) {
3448 if (processSingle) {
3449
1/6
✗ Branch 168 → 169 not taken.
✗ Branch 168 → 1159 not taken.
✗ Branch 7120 → 7121 not taken.
✗ Branch 7120 → 8111 not taken.
✓ Branch 8672 → 8673 taken 7 times.
✗ Branch 8672 → 9668 not taken.
 7 TwoArgOp_Single_Avx(vaddps);
3450 }
3451 else {
3452
2/8
✓ Branch 1757 → 1758 taken 7 times.
✗ Branch 1757 → 3487 not taken.
✓ Branch 1758 → 1759 taken 7 times.
✗ Branch 1758 → 3487 not taken.
✗ Branch 4487 → 4488 not taken.
✗ Branch 4487 → 6222 not taken.
✗ Branch 4488 → 4489 not taken.
✗ Branch 4488 → 6222 not taken.
 7 TwoArgOp_Avx(vaddps);
3453 }
3454 }
3455
2/10
✗ Branch 170 → 171 not taken.
✗ Branch 170 → 176 not taken.
✗ Branch 1760 → 1761 not taken.
✓ Branch 1760 → 1767 taken 6 times.
✗ Branch 4490 → 4491 not taken.
✗ Branch 4490 → 4497 not taken.
✗ Branch 7122 → 7123 not taken.
✗ Branch 7122 → 7128 not taken.
✗ Branch 8674 → 8675 not taken.
✓ Branch 8674 → 8680 taken 6 times.
 12 else if (iter.op == opSub) {
3456 if (processSingle) {
3457 ✗ TwoArgOp_Single_Avx(vsubps);
3458 }
3459 else {
3460 ✗ TwoArgOp_Avx(vsubps);
3461 }
3462 }
3463
2/10
✗ Branch 176 → 177 not taken.
✗ Branch 176 → 182 not taken.
✗ Branch 1767 → 1768 not taken.
✓ Branch 1767 → 1774 taken 6 times.
✗ Branch 4497 → 4498 not taken.
✗ Branch 4497 → 4504 not taken.
✗ Branch 7128 → 7129 not taken.
✗ Branch 7128 → 7134 not taken.
✗ Branch 8680 → 8681 not taken.
✓ Branch 8680 → 8686 taken 6 times.
 12 else if (iter.op == opMul) {
3464 if (processSingle) {
3465 ✗ TwoArgOp_Single_Avx(vmulps);
3466 }
3467 else {
3468 ✗ TwoArgOp_Avx(vmulps);
3469 }
3470 }
3471
2/10
✗ Branch 182 → 183 not taken.
✗ Branch 182 → 188 not taken.
✗ Branch 1774 → 1775 not taken.
✓ Branch 1774 → 1781 taken 6 times.
✗ Branch 4504 → 4505 not taken.
✗ Branch 4504 → 4511 not taken.
✗ Branch 7134 → 7135 not taken.
✗ Branch 7134 → 7140 not taken.
✗ Branch 8686 → 8687 not taken.
✓ Branch 8686 → 8692 taken 6 times.
 12 else if (iter.op == opDiv) {
3472 if (processSingle) {
3473 ✗ TwoArgOp_Single_Avx(vdivps);
3474 }
3475 else {
3476 ✗ TwoArgOp_Avx(vdivps);
3477 }
3478 }
3479
2/10
✗ Branch 188 → 189 not taken.
✗ Branch 188 → 199 not taken.
✗ Branch 1781 → 1782 not taken.
✓ Branch 1781 → 1798 taken 6 times.
✗ Branch 4511 → 4512 not taken.
✗ Branch 4511 → 4528 not taken.
✗ Branch 7140 → 7141 not taken.
✗ Branch 7140 → 7151 not taken.
✗ Branch 8692 → 8693 not taken.
✓ Branch 8692 → 8703 taken 6 times.
 12 else if (iter.op == opFmod) {
3480 if (processSingle) {
3481 ✗ auto t1 = stack1.back();
3482 ✗ stack1.pop_back();
3483 ✗ auto &t2 = stack1.back();
3484 ✗ FMOD_PS_AVX(t2, t1)
3485 }
3486 else {
3487 ✗ auto t1 = stack.back();
3488 ✗ stack.pop_back();
3489 ✗ auto &t2 = stack.back();
3490 ✗ FMOD_PS_AVX(t2.first, t1.first)
3491 ✗ FMOD_PS_AVX(t2.second, t1.second)
3492 }
3493 }
3494
2/10
✗ Branch 199 → 200 not taken.
✗ Branch 199 → 205 not taken.
✗ Branch 1798 → 1799 not taken.
✓ Branch 1798 → 1805 taken 6 times.
✗ Branch 4528 → 4529 not taken.
✗ Branch 4528 → 4535 not taken.
✗ Branch 7151 → 7152 not taken.
✗ Branch 7151 → 7157 not taken.
✗ Branch 8703 → 8704 not taken.
✓ Branch 8703 → 8709 taken 6 times.
 12 else if (iter.op == opMax) {
3495 if (processSingle) {
3496 ✗ TwoArgOp_Single_Avx(vmaxps);
3497 }
3498 else {
3499 ✗ TwoArgOp_Avx(vmaxps);
3500 }
3501 }
3502
2/10
✗ Branch 205 → 206 not taken.
✗ Branch 205 → 211 not taken.
✗ Branch 1805 → 1806 not taken.
✓ Branch 1805 → 1812 taken 6 times.
✗ Branch 4535 → 4536 not taken.
✗ Branch 4535 → 4542 not taken.
✗ Branch 7157 → 7158 not taken.
✗ Branch 7157 → 7163 not taken.
✗ Branch 8709 → 8710 not taken.
✓ Branch 8709 → 8715 taken 6 times.
 12 else if (iter.op == opMin) {
3503 if (processSingle) {
3504 ✗ TwoArgOp_Single_Avx(vminps);
3505 }
3506 else {
3507 ✗ TwoArgOp_Avx(vminps);
3508 }
3509 }
3510
2/10
✗ Branch 211 → 212 not taken.
✗ Branch 211 → 215 not taken.
✗ Branch 1812 → 1813 not taken.
✓ Branch 1812 → 1818 taken 6 times.
✗ Branch 4542 → 4543 not taken.
✗ Branch 4542 → 4548 not taken.
✗ Branch 7163 → 7164 not taken.
✗ Branch 7163 → 7167 not taken.
✗ Branch 8715 → 8716 not taken.
✓ Branch 8715 → 8719 taken 6 times.
 12 else if (iter.op == opSqrt) {
3511 if (processSingle) {
3512 ✗ auto &t1 = stack1.back();
3513 ✗ vmaxps(t1, t1, zero);
3514 ✗ vsqrtps(t1, t1);
3515 }
3516 else {
3517 ✗ auto &t1 = stack.back();
3518 ✗ vmaxps(t1.first, t1.first, zero);
3519 ✗ vmaxps(t1.second, t1.second, zero);
3520 ✗ vsqrtps(t1.first, t1.first);
3521 ✗ vsqrtps(t1.second, t1.second);
3522 }
3523 }
3524
4/10
✗ Branch 215 → 216 not taken.
✗ Branch 215 → 242 not taken.
✓ Branch 1818 → 1819 taken 3 times.
✓ Branch 1818 → 1859 taken 3 times.
✗ Branch 4548 → 4549 not taken.
✗ Branch 4548 → 4589 not taken.
✗ Branch 7167 → 7168 not taken.
✗ Branch 7167 → 7194 not taken.
✓ Branch 8719 → 8720 taken 3 times.
✓ Branch 8719 → 8746 taken 3 times.
 12 else if (iter.op == opStore8) {
3525 if (processSingle) {
3526 3 auto t1 = stack1.back();
3527 3 stack1.pop_back();
3528
1/6
✗ Branch 218 → 219 not taken.
✗ Branch 218 → 1177 not taken.
✗ Branch 7170 → 7171 not taken.
✗ Branch 7170 → 8129 not taken.
✓ Branch 8722 → 8723 taken 3 times.
✗ Branch 8722 → 9686 not taken.
 3 Reg a;
3529
2/12
✗ Branch 220 → 221 not taken.
✗ Branch 220 → 1167 not taken.
✗ Branch 221 → 222 not taken.
✗ Branch 221 → 1167 not taken.
✗ Branch 7172 → 7173 not taken.
✗ Branch 7172 → 8119 not taken.
✗ Branch 7173 → 7174 not taken.
✗ Branch 7173 → 8119 not taken.
✓ Branch 8724 → 8725 taken 3 times.
✗ Branch 8724 → 9676 not taken.
✓ Branch 8725 → 8726 taken 3 times.
✗ Branch 8725 → 9676 not taken.
 3 vaddps(t1, t1, CPTR_AVX(elfloat_half)); // rounder for truncate! no banker's rounding
3530
1/6
✗ Branch 222 → 223 not taken.
✗ Branch 222 → 1177 not taken.
✗ Branch 7174 → 7175 not taken.
✗ Branch 7174 → 8129 not taken.
✓ Branch 8726 → 8727 taken 3 times.
✗ Branch 8726 → 9686 not taken.
 3 vmaxps(t1, t1, zero);
3531
2/12
✗ Branch 224 → 225 not taken.
✗ Branch 224 → 1169 not taken.
✗ Branch 225 → 226 not taken.
✗ Branch 225 → 1169 not taken.
✗ Branch 7176 → 7177 not taken.
✗ Branch 7176 → 8121 not taken.
✗ Branch 7177 → 7178 not taken.
✗ Branch 7177 → 8121 not taken.
✓ Branch 8728 → 8729 taken 3 times.
✗ Branch 8728 → 9678 not taken.
✓ Branch 8729 → 8730 taken 3 times.
✗ Branch 8729 → 9678 not taken.
 3 vminps(t1, t1, CPTR_AVX(elstore8));
3532
2/12
✗ Branch 227 → 228 not taken.
✗ Branch 227 → 1171 not taken.
✗ Branch 228 → 229 not taken.
✗ Branch 228 → 1171 not taken.
✗ Branch 7179 → 7180 not taken.
✗ Branch 7179 → 8123 not taken.
✗ Branch 7180 → 7181 not taken.
✗ Branch 7180 → 8123 not taken.
✓ Branch 8731 → 8732 taken 3 times.
✗ Branch 8731 → 9680 not taken.
✓ Branch 8732 → 8733 taken 3 times.
✗ Branch 8732 → 9680 not taken.
 3 mov(a, ptr[regptrs]);
3533
1/6
✗ Branch 229 → 230 not taken.
✗ Branch 229 → 1177 not taken.
✗ Branch 7181 → 7182 not taken.
✗ Branch 7181 → 8129 not taken.
✓ Branch 8733 → 8734 taken 3 times.
✗ Branch 8733 → 9686 not taken.
 3 vcvttps2dq(t1, t1); // float to int32 no bankers rounding
3534
2/12
✗ Branch 230 → 231 not taken.
✗ Branch 230 → 1177 not taken.
✗ Branch 231 → 232 not taken.
✗ Branch 231 → 1177 not taken.
✗ Branch 7182 → 7183 not taken.
✗ Branch 7182 → 8129 not taken.
✗ Branch 7183 → 7184 not taken.
✗ Branch 7183 → 8129 not taken.
✓ Branch 8734 → 8735 taken 3 times.
✗ Branch 8734 → 9686 not taken.
✓ Branch 8735 → 8736 taken 3 times.
✗ Branch 8735 → 9686 not taken.
 3 XmmReg r1x, r2x;
3535 // 32 -> 16 bits from ymm 8 integers to xmm 8 words
3536 // first
3537
2/12
✗ Branch 232 → 233 not taken.
✗ Branch 232 → 1173 not taken.
✗ Branch 233 → 234 not taken.
✗ Branch 233 → 1173 not taken.
✗ Branch 7184 → 7185 not taken.
✗ Branch 7184 → 8125 not taken.
✗ Branch 7185 → 7186 not taken.
✗ Branch 7185 → 8125 not taken.
✓ Branch 8736 → 8737 taken 3 times.
✗ Branch 8736 → 9682 not taken.
✓ Branch 8737 → 8738 taken 3 times.
✗ Branch 8737 → 9682 not taken.
 3 vextracti128(r1x, t1, 0);
3538
2/12
✗ Branch 234 → 235 not taken.
✗ Branch 234 → 1174 not taken.
✗ Branch 235 → 236 not taken.
✗ Branch 235 → 1174 not taken.
✗ Branch 7186 → 7187 not taken.
✗ Branch 7186 → 8126 not taken.
✗ Branch 7187 → 7188 not taken.
✗ Branch 7187 → 8126 not taken.
✓ Branch 8738 → 8739 taken 3 times.
✗ Branch 8738 → 9683 not taken.
✓ Branch 8739 → 8740 taken 3 times.
✗ Branch 8739 → 9683 not taken.
 3 vextracti128(r2x, t1, 1);
3539
1/6
✗ Branch 236 → 237 not taken.
✗ Branch 236 → 1177 not taken.
✗ Branch 7188 → 7189 not taken.
✗ Branch 7188 → 8129 not taken.
✓ Branch 8740 → 8741 taken 3 times.
✗ Branch 8740 → 9686 not taken.
 3 vpackusdw(r1x, r1x, r2x); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
3540 // 16 -> 8 bits
3541
1/6
✗ Branch 237 → 238 not taken.
✗ Branch 237 → 1177 not taken.
✗ Branch 7189 → 7190 not taken.
✗ Branch 7189 → 8129 not taken.
✓ Branch 8741 → 8742 taken 3 times.
✗ Branch 8741 → 9686 not taken.
 3 vpackuswb(r1x, r1x, r1x); // _mm_packus_epi16: w3 w2 w1 w0 w3 w2 w1 w0
3542
2/12
✗ Branch 239 → 240 not taken.
✗ Branch 239 → 1175 not taken.
✗ Branch 240 → 241 not taken.
✗ Branch 240 → 1175 not taken.
✗ Branch 7191 → 7192 not taken.
✗ Branch 7191 → 8127 not taken.
✗ Branch 7192 → 7193 not taken.
✗ Branch 7192 → 8127 not taken.
✓ Branch 8743 → 8744 taken 3 times.
✗ Branch 8743 → 9684 not taken.
✓ Branch 8744 → 8745 taken 3 times.
✗ Branch 8744 → 9684 not taken.
 3 vmovq(mmword_ptr[a], r1x); // store 8 bytes
3543 }
3544 else {
3545 3 auto t1 = stack.back();
3546 3 stack.pop_back();
3547
1/4
✓ Branch 1821 → 1822 taken 3 times.
✗ Branch 1821 → 3512 not taken.
✗ Branch 4551 → 4552 not taken.
✗ Branch 4551 → 6247 not taken.
 3 Reg a;
3548
2/8
✓ Branch 1823 → 1824 taken 3 times.
✗ Branch 1823 → 3496 not taken.
✓ Branch 1824 → 1825 taken 3 times.
✗ Branch 1824 → 3496 not taken.
✗ Branch 4553 → 4554 not taken.
✗ Branch 4553 → 6231 not taken.
✗ Branch 4554 → 4555 not taken.
✗ Branch 4554 → 6231 not taken.
 3 vaddps(t1.first, t1.first, CPTR_AVX(elfloat_half)); // rounder for truncate! no banker's rounding
3549
1/4
✓ Branch 1825 → 1826 taken 3 times.
✗ Branch 1825 → 3512 not taken.
✗ Branch 4555 → 4556 not taken.
✗ Branch 4555 → 6247 not taken.
 3 vmaxps(t1.first, t1.first, zero);
3550
2/8
✓ Branch 1827 → 1828 taken 3 times.
✗ Branch 1827 → 3498 not taken.
✓ Branch 1828 → 1829 taken 3 times.
✗ Branch 1828 → 3498 not taken.
✗ Branch 4557 → 4558 not taken.
✗ Branch 4557 → 6233 not taken.
✗ Branch 4558 → 4559 not taken.
✗ Branch 4558 → 6233 not taken.
 3 vaddps(t1.second, t1.second, CPTR_AVX(elfloat_half)); // rounder for truncate! no banker's rounding
3551
1/4
✓ Branch 1829 → 1830 taken 3 times.
✗ Branch 1829 → 3512 not taken.
✗ Branch 4559 → 4560 not taken.
✗ Branch 4559 → 6247 not taken.
 3 vmaxps(t1.second, t1.second, zero);
3552
2/8
✓ Branch 1831 → 1832 taken 3 times.
✗ Branch 1831 → 3500 not taken.
✓ Branch 1832 → 1833 taken 3 times.
✗ Branch 1832 → 3500 not taken.
✗ Branch 4561 → 4562 not taken.
✗ Branch 4561 → 6235 not taken.
✗ Branch 4562 → 4563 not taken.
✗ Branch 4562 → 6235 not taken.
 3 vminps(t1.first, t1.first, CPTR_AVX(elstore8));
3553
2/8
✓ Branch 1834 → 1835 taken 3 times.
✗ Branch 1834 → 3502 not taken.
✓ Branch 1835 → 1836 taken 3 times.
✗ Branch 1835 → 3502 not taken.
✗ Branch 4564 → 4565 not taken.
✗ Branch 4564 → 6237 not taken.
✗ Branch 4565 → 4566 not taken.
✗ Branch 4565 → 6237 not taken.
 3 vminps(t1.second, t1.second, CPTR_AVX(elstore8));
3554
2/8
✓ Branch 1837 → 1838 taken 3 times.
✗ Branch 1837 → 3504 not taken.
✓ Branch 1838 → 1839 taken 3 times.
✗ Branch 1838 → 3504 not taken.
✗ Branch 4567 → 4568 not taken.
✗ Branch 4567 → 6239 not taken.
✗ Branch 4568 → 4569 not taken.
✗ Branch 4568 → 6239 not taken.
 3 mov(a, ptr[regptrs]);
3555
1/4
✓ Branch 1839 → 1840 taken 3 times.
✗ Branch 1839 → 3512 not taken.
✗ Branch 4569 → 4570 not taken.
✗ Branch 4569 → 6247 not taken.
 3 vcvttps2dq(t1.first, t1.first); // float to int32 no bankers rounding
3556
1/4
✓ Branch 1840 → 1841 taken 3 times.
✗ Branch 1840 → 3512 not taken.
✗ Branch 4570 → 4571 not taken.
✗ Branch 4570 → 6247 not taken.
 3 vcvttps2dq(t1.second, t1.second);
3557 // we have 8 integers in t.first and another 8 in t.second
3558 // second first
3559 // d15 d14 d13 d12 d11 d10 d9 d8 d7 d6 d5 d4 d3 d2 d1 d0 // 16x32 bit integers in two ymm registers. not really 256 bits, but 2x128 bits
3560
3/12
✓ Branch 1841 → 1842 taken 3 times.
✗ Branch 1841 → 3512 not taken.
✓ Branch 1842 → 1843 taken 3 times.
✗ Branch 1842 → 3512 not taken.
✓ Branch 1843 → 1844 taken 3 times.
✗ Branch 1843 → 3512 not taken.
✗ Branch 4571 → 4572 not taken.
✗ Branch 4571 → 6247 not taken.
✗ Branch 4572 → 4573 not taken.
✗ Branch 4572 → 6247 not taken.
✗ Branch 4573 → 4574 not taken.
✗ Branch 4573 → 6247 not taken.
 3 XmmReg r1x, r2x, r_lo_x;
3561 // 32 -> 16 bits from ymm 8 integers to xmm 8 words
3562 // first
3563
2/8
✓ Branch 1844 → 1845 taken 3 times.
✗ Branch 1844 → 3506 not taken.
✓ Branch 1845 → 1846 taken 3 times.
✗ Branch 1845 → 3506 not taken.
✗ Branch 4574 → 4575 not taken.
✗ Branch 4574 → 6241 not taken.
✗ Branch 4575 → 4576 not taken.
✗ Branch 4575 → 6241 not taken.
 3 vextracti128(r1x, t1.first, 0);
3564
2/8
✓ Branch 1846 → 1847 taken 3 times.
✗ Branch 1846 → 3507 not taken.
✓ Branch 1847 → 1848 taken 3 times.
✗ Branch 1847 → 3507 not taken.
✗ Branch 4576 → 4577 not taken.
✗ Branch 4576 → 6242 not taken.
✗ Branch 4577 → 4578 not taken.
✗ Branch 4577 → 6242 not taken.
 3 vextracti128(r2x, t1.first, 1);
3565
1/4
✓ Branch 1848 → 1849 taken 3 times.
✗ Branch 1848 → 3512 not taken.
✗ Branch 4578 → 4579 not taken.
✗ Branch 4578 → 6247 not taken.
 3 vpackusdw(r_lo_x, r1x, r2x); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
3566 // second
3567
2/8
✓ Branch 1849 → 1850 taken 3 times.
✗ Branch 1849 → 3508 not taken.
✓ Branch 1850 → 1851 taken 3 times.
✗ Branch 1850 → 3508 not taken.
✗ Branch 4579 → 4580 not taken.
✗ Branch 4579 → 6243 not taken.
✗ Branch 4580 → 4581 not taken.
✗ Branch 4580 → 6243 not taken.
 3 vextracti128(r1x, t1.second, 0); // not perfect, lower 128 bits of t1 could be used as xmm in packus. Cannot tell jitasm that xxmN is lower ymmN
3568
2/8
✓ Branch 1851 → 1852 taken 3 times.
✗ Branch 1851 → 3509 not taken.
✓ Branch 1852 → 1853 taken 3 times.
✗ Branch 1852 → 3509 not taken.
✗ Branch 4581 → 4582 not taken.
✗ Branch 4581 → 6244 not taken.
✗ Branch 4582 → 4583 not taken.
✗ Branch 4582 → 6244 not taken.
 3 vextracti128(r2x, t1.second, 1);
3569
1/4
✓ Branch 1853 → 1854 taken 3 times.
✗ Branch 1853 → 3512 not taken.
✗ Branch 4583 → 4584 not taken.
✗ Branch 4583 → 6247 not taken.
 3 vpackusdw(r1x, r1x, r2x); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
3570 // 16 -> 8 bits
3571
1/4
✓ Branch 1854 → 1855 taken 3 times.
✗ Branch 1854 → 3512 not taken.
✗ Branch 4584 → 4585 not taken.
✗ Branch 4584 → 6247 not taken.
 3 vpackuswb(r1x, r_lo_x, r1x); // _mm_packus_epi16: w3 w2 w1 w0 w3 w2 w1 w0
3572
2/8
✓ Branch 1856 → 1857 taken 3 times.
✗ Branch 1856 → 3510 not taken.
✓ Branch 1857 → 1858 taken 3 times.
✗ Branch 1857 → 3510 not taken.
✗ Branch 4586 → 4587 not taken.
✗ Branch 4586 → 6245 not taken.
✗ Branch 4587 → 4588 not taken.
✗ Branch 4587 → 6245 not taken.
 3 vmovdqa(xmmword_ptr[a], r1x); // store 16 bytes
3573 }
3574 }
3575
2/10
✗ Branch 242 → 243 not taken.
✗ Branch 242 → 246 not taken.
✓ Branch 1859 → 1860 taken 3 times.
✗ Branch 1859 → 1863 not taken.
✗ Branch 4589 → 4590 not taken.
✗ Branch 4589 → 4593 not taken.
✗ Branch 7194 → 7195 not taken.
✗ Branch 7194 → 7198 not taken.
✓ Branch 8746 → 8747 taken 3 times.
✗ Branch 8746 → 8750 not taken.
 6 else if (iter.op == opStore10 // avs+
3576
2/10
✗ Branch 243 → 244 not taken.
✗ Branch 243 → 246 not taken.
✓ Branch 1860 → 1861 taken 3 times.
✗ Branch 1860 → 1863 not taken.
✗ Branch 4590 → 4591 not taken.
✗ Branch 4590 → 4593 not taken.
✗ Branch 7195 → 7196 not taken.
✗ Branch 7195 → 7198 not taken.
✓ Branch 8747 → 8748 taken 3 times.
✗ Branch 8747 → 8750 not taken.
 6 || iter.op == opStore12 // avs+
3577
2/10
✗ Branch 244 → 245 not taken.
✗ Branch 244 → 246 not taken.
✓ Branch 1861 → 1862 taken 3 times.
✗ Branch 1861 → 1863 not taken.
✗ Branch 4591 → 4592 not taken.
✗ Branch 4591 → 4593 not taken.
✗ Branch 7196 → 7197 not taken.
✗ Branch 7196 → 7198 not taken.
✓ Branch 8748 → 8749 taken 3 times.
✗ Branch 8748 → 8750 not taken.
 6 || iter.op == opStore14 // avs+
3578
2/10
✗ Branch 245 → 246 not taken.
✗ Branch 245 → 286 not taken.
✓ Branch 1862 → 1863 taken 3 times.
✗ Branch 1862 → 1927 not taken.
✗ Branch 4592 → 4593 not taken.
✗ Branch 4592 → 4657 not taken.
✗ Branch 7197 → 7198 not taken.
✗ Branch 7197 → 7238 not taken.
✓ Branch 8749 → 8750 taken 3 times.
✗ Branch 8749 → 8790 not taken.
 6 || iter.op == opStore16
3579 ) {
3580 if (processSingle) {
3581 3 auto t1 = stack1.back();
3582 3 stack1.pop_back();
3583
1/6
✗ Branch 248 → 249 not taken.
✗ Branch 248 → 1194 not taken.
✗ Branch 7200 → 7201 not taken.
✗ Branch 7200 → 8146 not taken.
✓ Branch 8752 → 8753 taken 3 times.
✗ Branch 8752 → 9703 not taken.
 3 Reg a;
3584
2/12
✗ Branch 250 → 251 not taken.
✗ Branch 250 → 1178 not taken.
✗ Branch 251 → 252 not taken.
✗ Branch 251 → 1178 not taken.
✗ Branch 7202 → 7203 not taken.
✗ Branch 7202 → 8130 not taken.
✗ Branch 7203 → 7204 not taken.
✗ Branch 7203 → 8130 not taken.
✓ Branch 8754 → 8755 taken 3 times.
✗ Branch 8754 → 9687 not taken.
✓ Branch 8755 → 8756 taken 3 times.
✗ Branch 8755 → 9687 not taken.
 3 vaddps(t1, t1, CPTR_AVX(elfloat_half)); // rounder for truncate! no banker's rounding
3585
1/6
✗ Branch 252 → 253 not taken.
✗ Branch 252 → 1194 not taken.
✗ Branch 7204 → 7205 not taken.
✗ Branch 7204 → 8146 not taken.
✓ Branch 8756 → 8757 taken 3 times.
✗ Branch 8756 → 9703 not taken.
 3 vmaxps(t1, t1, zero);
3586
1/15
✗ Branch 253 → 254 not taken.
✗ Branch 253 → 258 not taken.
✗ Branch 253 → 262 not taken.
✗ Branch 253 → 266 not taken.
✗ Branch 253 → 270 not taken.
✗ Branch 7205 → 7206 not taken.
✗ Branch 7205 → 7210 not taken.
✗ Branch 7205 → 7214 not taken.
✗ Branch 7205 → 7218 not taken.
✗ Branch 7205 → 7222 not taken.
✗ Branch 8757 → 8758 not taken.
✗ Branch 8757 → 8762 not taken.
✗ Branch 8757 → 8766 not taken.
✓ Branch 8757 → 8770 taken 3 times.
✗ Branch 8757 → 8774 not taken.
 3 switch (iter.op) {
3587 ✗ case opStore10:
3588 ✗ vminps(t1, t1, CPTR_AVX(elstore10));
3589 ✗ break;
3590 ✗ case opStore12:
3591 ✗ vminps(t1, t1, CPTR_AVX(elstore12));
3592 ✗ break;
3593 ✗ case opStore14:
3594 ✗ vminps(t1, t1, CPTR_AVX(elstore14));
3595 ✗ break;
3596 3 case opStore16:
3597
2/12
✗ Branch 267 → 268 not taken.
✗ Branch 267 → 1186 not taken.
✗ Branch 268 → 269 not taken.
✗ Branch 268 → 1186 not taken.
✗ Branch 7219 → 7220 not taken.
✗ Branch 7219 → 8138 not taken.
✗ Branch 7220 → 7221 not taken.
✗ Branch 7220 → 8138 not taken.
✓ Branch 8771 → 8772 taken 3 times.
✗ Branch 8771 → 9695 not taken.
✓ Branch 8772 → 8773 taken 3 times.
✗ Branch 8772 → 9695 not taken.
 3 vminps(t1, t1, CPTR_AVX(elstore16));
3598 3 break;
3599 }
3600
2/12
✗ Branch 271 → 272 not taken.
✗ Branch 271 → 1188 not taken.
✗ Branch 272 → 273 not taken.
✗ Branch 272 → 1188 not taken.
✗ Branch 7223 → 7224 not taken.
✗ Branch 7223 → 8140 not taken.
✗ Branch 7224 → 7225 not taken.
✗ Branch 7224 → 8140 not taken.
✓ Branch 8775 → 8776 taken 3 times.
✗ Branch 8775 → 9697 not taken.
✓ Branch 8776 → 8777 taken 3 times.
✗ Branch 8776 → 9697 not taken.
 3 mov(a, ptr[regptrs]);
3601
1/6
✗ Branch 273 → 274 not taken.
✗ Branch 273 → 1194 not taken.
✗ Branch 7225 → 7226 not taken.
✗ Branch 7225 → 8146 not taken.
✓ Branch 8777 → 8778 taken 3 times.
✗ Branch 8777 → 9703 not taken.
 3 vcvttps2dq(t1, t1); // min / max clamp ensures that high words are zero
3602
2/12
✗ Branch 274 → 275 not taken.
✗ Branch 274 → 1194 not taken.
✗ Branch 275 → 276 not taken.
✗ Branch 275 → 1194 not taken.
✗ Branch 7226 → 7227 not taken.
✗ Branch 7226 → 8146 not taken.
✗ Branch 7227 → 7228 not taken.
✗ Branch 7227 → 8146 not taken.
✓ Branch 8778 → 8779 taken 3 times.
✗ Branch 8778 → 9703 not taken.
✓ Branch 8779 → 8780 taken 3 times.
✗ Branch 8779 → 9703 not taken.
 3 XmmReg r1x, r2x;
3603 // 32 -> 16 bits from ymm 8 integers to xmm 8 words
3604
2/12
✗ Branch 276 → 277 not taken.
✗ Branch 276 → 1190 not taken.
✗ Branch 277 → 278 not taken.
✗ Branch 277 → 1190 not taken.
✗ Branch 7228 → 7229 not taken.
✗ Branch 7228 → 8142 not taken.
✗ Branch 7229 → 7230 not taken.
✗ Branch 7229 → 8142 not taken.
✓ Branch 8780 → 8781 taken 3 times.
✗ Branch 8780 → 9699 not taken.
✓ Branch 8781 → 8782 taken 3 times.
✗ Branch 8781 → 9699 not taken.
 3 vextracti128(r1x, t1, 0); // not perfect, lower 128 bits of t1 could be used as xmm in packus. Cannot tell jitasm that xxmN is lower ymmN
3605
2/12
✗ Branch 278 → 279 not taken.
✗ Branch 278 → 1191 not taken.
✗ Branch 279 → 280 not taken.
✗ Branch 279 → 1191 not taken.
✗ Branch 7230 → 7231 not taken.
✗ Branch 7230 → 8143 not taken.
✗ Branch 7231 → 7232 not taken.
✗ Branch 7231 → 8143 not taken.
✓ Branch 8782 → 8783 taken 3 times.
✗ Branch 8782 → 9700 not taken.
✓ Branch 8783 → 8784 taken 3 times.
✗ Branch 8783 → 9700 not taken.
 3 vextracti128(r2x, t1, 1);
3606
1/6
✗ Branch 280 → 281 not taken.
✗ Branch 280 → 1194 not taken.
✗ Branch 7232 → 7233 not taken.
✗ Branch 7232 → 8146 not taken.
✓ Branch 8784 → 8785 taken 3 times.
✗ Branch 8784 → 9703 not taken.
 3 vpackusdw(r1x, r1x, r2x); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
3607
2/12
✗ Branch 282 → 283 not taken.
✗ Branch 282 → 1192 not taken.
✗ Branch 283 → 284 not taken.
✗ Branch 283 → 1192 not taken.
✗ Branch 7234 → 7235 not taken.
✗ Branch 7234 → 8144 not taken.
✗ Branch 7235 → 7236 not taken.
✗ Branch 7235 → 8144 not taken.
✓ Branch 8786 → 8787 taken 3 times.
✗ Branch 8786 → 9701 not taken.
✓ Branch 8787 → 8788 taken 3 times.
✗ Branch 8787 → 9701 not taken.
 3 vmovdqa(xmmword_ptr[a], r1x);
3608 }
3609 else {
3610 3 auto t1 = stack.back();
3611 3 stack.pop_back();
3612
1/4
✓ Branch 1865 → 1866 taken 3 times.
✗ Branch 1865 → 3543 not taken.
✗ Branch 4595 → 4596 not taken.
✗ Branch 4595 → 6278 not taken.
 3 Reg a;
3613
2/8
✓ Branch 1867 → 1868 taken 3 times.
✗ Branch 1867 → 3513 not taken.
✓ Branch 1868 → 1869 taken 3 times.
✗ Branch 1868 → 3513 not taken.
✗ Branch 4597 → 4598 not taken.
✗ Branch 4597 → 6248 not taken.
✗ Branch 4598 → 4599 not taken.
✗ Branch 4598 → 6248 not taken.
 3 vaddps(t1.first, t1.first, CPTR_AVX(elfloat_half)); // rounder for truncate! no banker's rounding
3614
1/4
✓ Branch 1869 → 1870 taken 3 times.
✗ Branch 1869 → 3543 not taken.
✗ Branch 4599 → 4600 not taken.
✗ Branch 4599 → 6278 not taken.
 3 vmaxps(t1.first, t1.first, zero);
3615
2/8
✓ Branch 1871 → 1872 taken 3 times.
✗ Branch 1871 → 3515 not taken.
✓ Branch 1872 → 1873 taken 3 times.
✗ Branch 1872 → 3515 not taken.
✗ Branch 4601 → 4602 not taken.
✗ Branch 4601 → 6250 not taken.
✗ Branch 4602 → 4603 not taken.
✗ Branch 4602 → 6250 not taken.
 3 vaddps(t1.second, t1.second, CPTR_AVX(elfloat_half)); // rounder for truncate! no banker's rounding
3616
1/4
✓ Branch 1873 → 1874 taken 3 times.
✗ Branch 1873 → 3543 not taken.
✗ Branch 4603 → 4604 not taken.
✗ Branch 4603 → 6278 not taken.
 3 vmaxps(t1.second, t1.second, zero);
3617
1/10
✗ Branch 1874 → 1875 not taken.
✗ Branch 1874 → 1882 not taken.
✗ Branch 1874 → 1889 not taken.
✓ Branch 1874 → 1896 taken 3 times.
✗ Branch 1874 → 1903 not taken.
✗ Branch 4604 → 4605 not taken.
✗ Branch 4604 → 4612 not taken.
✗ Branch 4604 → 4619 not taken.
✗ Branch 4604 → 4626 not taken.
✗ Branch 4604 → 4633 not taken.
 3 switch (iter.op) {
3618 ✗ case opStore10:
3619 ✗ vminps(t1.first, t1.first, CPTR_AVX(elstore10));
3620 ✗ vminps(t1.second, t1.second, CPTR_AVX(elstore10));
3621 ✗ break;
3622 ✗ case opStore12:
3623 ✗ vminps(t1.first, t1.first, CPTR_AVX(elstore12));
3624 ✗ vminps(t1.second, t1.second, CPTR_AVX(elstore12));
3625 ✗ break;
3626 ✗ case opStore14:
3627 ✗ vminps(t1.first, t1.first, CPTR_AVX(elstore14));
3628 ✗ vminps(t1.second, t1.second, CPTR_AVX(elstore14));
3629 ✗ break;
3630 3 case opStore16:
3631
2/8
✓ Branch 1897 → 1898 taken 3 times.
✗ Branch 1897 → 3529 not taken.
✓ Branch 1898 → 1899 taken 3 times.
✗ Branch 1898 → 3529 not taken.
✗ Branch 4627 → 4628 not taken.
✗ Branch 4627 → 6264 not taken.
✗ Branch 4628 → 4629 not taken.
✗ Branch 4628 → 6264 not taken.
 3 vminps(t1.first, t1.first, CPTR_AVX(elstore16));
3632
2/8
✓ Branch 1900 → 1901 taken 3 times.
✗ Branch 1900 → 3531 not taken.
✓ Branch 1901 → 1902 taken 3 times.
✗ Branch 1901 → 3531 not taken.
✗ Branch 4630 → 4631 not taken.
✗ Branch 4630 → 6266 not taken.
✗ Branch 4631 → 4632 not taken.
✗ Branch 4631 → 6266 not taken.
 3 vminps(t1.second, t1.second, CPTR_AVX(elstore16));
3633 3 break;
3634 }
3635
2/8
✓ Branch 1904 → 1905 taken 3 times.
✗ Branch 1904 → 3533 not taken.
✓ Branch 1905 → 1906 taken 3 times.
✗ Branch 1905 → 3533 not taken.
✗ Branch 4634 → 4635 not taken.
✗ Branch 4634 → 6268 not taken.
✗ Branch 4635 → 4636 not taken.
✗ Branch 4635 → 6268 not taken.
 3 mov(a, ptr[regptrs]);
3636
1/4
✓ Branch 1906 → 1907 taken 3 times.
✗ Branch 1906 → 3543 not taken.
✗ Branch 4636 → 4637 not taken.
✗ Branch 4636 → 6278 not taken.
 3 vcvttps2dq(t1.first, t1.first); // min / max clamp ensures that high words are zero
3637
1/4
✓ Branch 1907 → 1908 taken 3 times.
✗ Branch 1907 → 3543 not taken.
✗ Branch 4637 → 4638 not taken.
✗ Branch 4637 → 6278 not taken.
 3 vcvttps2dq(t1.second, t1.second);
3638 // we have 8 integers in t.first and another 8 in t.second
3639 // second first
3640 // d15 d14 d13 d12 d11 d10 d9 d8 d7 d6 d5 d4 d3 d2 d1 d0 // 16x32 bit integers in two ymm registers. not really 256 bits, but 2x128 bits
3641
2/8
✓ Branch 1908 → 1909 taken 3 times.
✗ Branch 1908 → 3543 not taken.
✓ Branch 1909 → 1910 taken 3 times.
✗ Branch 1909 → 3543 not taken.
✗ Branch 4638 → 4639 not taken.
✗ Branch 4638 → 6278 not taken.
✗ Branch 4639 → 4640 not taken.
✗ Branch 4639 → 6278 not taken.
 3 XmmReg r1x, r2x;
3642 // 32 -> 16 bits from ymm 8 integers to xmm 8 words
3643 // first
3644
2/8
✓ Branch 1910 → 1911 taken 3 times.
✗ Branch 1910 → 3535 not taken.
✓ Branch 1911 → 1912 taken 3 times.
✗ Branch 1911 → 3535 not taken.
✗ Branch 4640 → 4641 not taken.
✗ Branch 4640 → 6270 not taken.
✗ Branch 4641 → 4642 not taken.
✗ Branch 4641 → 6270 not taken.
 3 vextracti128(r1x, t1.first, 0); // not perfect, lower 128 bits of t1 could be used as xmm in packus. Cannot tell jitasm that xxmN is lower ymmN
3645
2/8
✓ Branch 1912 → 1913 taken 3 times.
✗ Branch 1912 → 3536 not taken.
✓ Branch 1913 → 1914 taken 3 times.
✗ Branch 1913 → 3536 not taken.
✗ Branch 4642 → 4643 not taken.
✗ Branch 4642 → 6271 not taken.
✗ Branch 4643 → 4644 not taken.
✗ Branch 4643 → 6271 not taken.
 3 vextracti128(r2x, t1.first, 1);
3646
1/4
✓ Branch 1914 → 1915 taken 3 times.
✗ Branch 1914 → 3543 not taken.
✗ Branch 4644 → 4645 not taken.
✗ Branch 4644 → 6278 not taken.
 3 vpackusdw(r1x, r1x, r2x); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
3647
2/8
✓ Branch 1916 → 1917 taken 3 times.
✗ Branch 1916 → 3537 not taken.
✓ Branch 1917 → 1918 taken 3 times.
✗ Branch 1917 → 3537 not taken.
✗ Branch 4646 → 4647 not taken.
✗ Branch 4646 → 6272 not taken.
✗ Branch 4647 → 4648 not taken.
✗ Branch 4647 → 6272 not taken.
 3 vmovdqa(xmmword_ptr[a], r1x);
3648 // second
3649
2/8
✓ Branch 1918 → 1919 taken 3 times.
✗ Branch 1918 → 3539 not taken.
✓ Branch 1919 → 1920 taken 3 times.
✗ Branch 1919 → 3539 not taken.
✗ Branch 4648 → 4649 not taken.
✗ Branch 4648 → 6274 not taken.
✗ Branch 4649 → 4650 not taken.
✗ Branch 4649 → 6274 not taken.
 3 vextracti128(r1x, t1.second, 0);
3650
2/8
✓ Branch 1920 → 1921 taken 3 times.
✗ Branch 1920 → 3540 not taken.
✓ Branch 1921 → 1922 taken 3 times.
✗ Branch 1921 → 3540 not taken.
✗ Branch 4650 → 4651 not taken.
✗ Branch 4650 → 6275 not taken.
✗ Branch 4651 → 4652 not taken.
✗ Branch 4651 → 6275 not taken.
 3 vextracti128(r2x, t1.second, 1);
3651
1/4
✓ Branch 1922 → 1923 taken 3 times.
✗ Branch 1922 → 3543 not taken.
✗ Branch 4652 → 4653 not taken.
✗ Branch 4652 → 6278 not taken.
 3 vpackusdw(r1x, r1x, r2x); // _mm_packus_epi32: w7 w6 w5 w4 w3 w2 w1 w0
3652
2/8
✓ Branch 1924 → 1925 taken 3 times.
✗ Branch 1924 → 3541 not taken.
✓ Branch 1925 → 1926 taken 3 times.
✗ Branch 1925 → 3541 not taken.
✗ Branch 4654 → 4655 not taken.
✗ Branch 4654 → 6276 not taken.
✗ Branch 4655 → 4656 not taken.
✗ Branch 4655 → 6276 not taken.
 3 vmovdqa(xmmword_ptr[a + 16], r1x);
3653 }
3654 6 }
3655 ✗ else if (iter.op == opStoreF32) {
3656 if (processSingle) {
3657 ✗ auto t1 = stack1.back();
3658 ✗ stack1.pop_back();
3659 ✗ Reg a;
3660 ✗ mov(a, ptr[regptrs]);
3661 ✗ vmovaps(ymmword_ptr[a], t1);
3662 } else {
3663 ✗ auto t1 = stack.back();
3664 ✗ stack.pop_back();
3665 ✗ Reg a;
3666 ✗ mov(a, ptr[regptrs]);
3667 ✗ vmovaps(ymmword_ptr[a], t1.first);
3668 ✗ vmovaps(ymmword_ptr[a + 32], t1.second); // this needs 64 byte aligned data to prevent overwrite!
3669 }
3670 }
3671 ✗ else if (iter.op == opStoreF16) { // not supported in avs+
3672 if (processSingle) {
3673 ✗ auto t1 = stack1.back();
3674 ✗ stack1.pop_back();
3675 ✗ Reg a;
3676 ✗ mov(a, ptr[regptrs]);
3677 ✗ vcvtps2ph(xmmword_ptr[a], t1, 0);
3678 } else {
3679 ✗ auto t1 = stack.back();
3680 ✗ stack.pop_back();
3681 ✗ Reg a;
3682 ✗ mov(a, ptr[regptrs]);
3683 ✗ vcvtps2ph(xmmword_ptr[a], t1.first, 0);
3684 ✗ vcvtps2ph(xmmword_ptr[a + 16], t1.second, 0);
3685 }
3686 }
3687 ✗ else if (iter.op == opStoreVar || iter.op == opStoreVarAndDrop1) {
3688 if (processSingle) {
3689 ✗ auto t1 = stack1.back();
3690 // 32 bytes/variable
3691 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 32 * iter.e.ival;
3692 ✗ vmovaps(ymmword_ptr[regptrs + offset], t1);
3693 ✗ if (iter.op == opStoreVarAndDrop1)
3694 ✗ stack1.pop_back();
3695 }
3696 else {
3697 ✗ auto t1 = stack.back();
3698 // 64 bytes/variable
3699 ✗ int offset = sizeof(void *) * RWPTR_START_OF_USERVARIABLES + 64 * iter.e.ival;
3700 ✗ vmovaps(ymmword_ptr[regptrs + offset], t1.first);
3701 ✗ vmovaps(ymmword_ptr[regptrs + offset + 32], t1.second); // this needs 64 byte aligned data to prevent overwrite!
3702 ✗ if (iter.op == opStoreVarAndDrop1)
3703 ✗ stack.pop_back();
3704 }
3705 ✗ }
3706 ✗ else if (iter.op == opAbs) {
3707 if (processSingle) {
3708 ✗ auto &t1 = stack1.back();
3709 ✗ vandps(t1, t1, CPTR_AVX(elabsmask));
3710 }
3711 else {
3712 ✗ auto &t1 = stack.back();
3713 ✗ vandps(t1.first, t1.first, CPTR_AVX(elabsmask));
3714 ✗ vandps(t1.second, t1.second, CPTR_AVX(elabsmask));
3715 }
3716 }
3717 ✗ else if (iter.op == opSgn) {
3718 // 1, 0, -1
3719 if (processSingle) {
3720 ✗ auto &t1 = stack1.back();
3721 ✗ YmmReg r1, r2;
3722 ✗ vxorps(r2, r2, r2);
3723 ✗ vcmpps(r1, t1, r2, _CMP_GT_OQ);
3724 ✗ vcmpps(t1, t1, r2, _CMP_LT_OQ);
3725 ✗ vandps(r1, r1, CPTR_AVX(elfloat_one));
3726 ✗ vandps(t1, t1, CPTR_AVX(elfloat_minusone));
3727 ✗ vorps(t1, r1, t1);
3728 }
3729 else {
3730 ✗ auto &t1 = stack.back();
3731 ✗ YmmReg r2, r3, r4, r5;
3732 ✗ vxorps(r2, r2, r2);
3733 ✗ vcmpps(r3, t1.first, r2, _CMP_GT_OQ);
3734 ✗ vcmpps(t1.first, t1.first, r2, _CMP_LT_OQ);
3735 ✗ vcmpps(r4, t1.second, r2, _CMP_GT_OQ);
3736 ✗ vcmpps(t1.second, t1.second, r2, _CMP_LT_OQ);
3737 ✗ vmovaps(r2, CPTR_AVX(elfloat_one));
3738 ✗ vandps(r3, r3, r2);
3739 ✗ vmovaps(r5, CPTR_AVX(elfloat_minusone));
3740 ✗ vblendvps(t1.first, r3, r5, t1.first);
3741 ✗ vandps(r2, r4, r2);
3742 ✗ vblendvps(t1.second, r2, r5, t1.second);
3743 }
3744 }
3745 ✗ else if (iter.op == opNeg) {
3746 if (processSingle) {
3747 ✗ auto &t1 = stack1.back();
3748 ✗ vcmpps(t1, t1, zero, _CMP_LE_OQ); // cmpleps
3749 ✗ vandps(t1, t1, CPTR_AVX(elfloat_one));
3750 }
3751 else {
3752 ✗ auto &t1 = stack.back();
3753 ✗ vcmpps(t1.first, t1.first, zero, _CMP_LE_OQ); // cmpleps
3754 ✗ vcmpps(t1.second, t1.second, zero, _CMP_LE_OQ);
3755 ✗ vandps(t1.first, t1.first, CPTR_AVX(elfloat_one));
3756 ✗ vandps(t1.second, t1.second, CPTR_AVX(elfloat_one));
3757 }
3758 }
3759 ✗ else if (iter.op == opNegSign) {
3760 if (processSingle) {
3761 ✗ auto& t1 = stack1.back();
3762 ✗ vxorps(t1, t1, CPTR_AVX(elsignmask));
3763 }
3764 else {
3765 ✗ auto& t1 = stack.back();
3766 ✗ vxorps(t1.first, t1.first, CPTR_AVX(elsignmask));
3767 ✗ vxorps(t1.second, t1.second, CPTR_AVX(elsignmask));
3768 }
3769 }
3770 ✗ else if (iter.op == opAnd) {
3771 if (processSingle) {
3772 ✗ LogicOp_Single_Avx(vandps);
3773 }
3774 else {
3775 ✗ LogicOp_Avx(vandps);
3776 }
3777 }
3778 ✗ else if (iter.op == opOr) {
3779 if (processSingle) {
3780 ✗ LogicOp_Single_Avx(vorps);
3781 }
3782 else {
3783 ✗ LogicOp_Avx(vorps);
3784 }
3785 }
3786 ✗ else if (iter.op == opXor) {
3787 if (processSingle) {
3788 ✗ LogicOp_Single_Avx(vxorps);
3789 }
3790 else {
3791 ✗ LogicOp_Avx(vxorps);
3792 }
3793 }
3794 ✗ else if (iter.op == opGt) { // a > b (gt) -> b < (lt) a
3795 if (processSingle) {
3796 ✗ CmpOp_Single_Avx(vcmpps, _CMP_LT_OQ); // cmpltps
3797 }
3798 else {
3799 ✗ CmpOp_Avx(vcmpps, _CMP_LT_OQ) // cmpltps
3800 }
3801 }
3802 ✗ else if (iter.op == opLt) { // a < b (lt) -> b > (gt,nle) a
3803 if (processSingle) {
3804 ✗ CmpOp_Single_Avx(vcmpps, _CMP_GT_OQ); // cmpnleps
3805 }
3806 else {
3807 ✗ CmpOp_Avx(vcmpps, _CMP_GT_OQ); // cmpnleps
3808 }
3809 }
3810 ✗ else if (iter.op == opEq) {
3811 if (processSingle) {
3812 ✗ CmpOp_Single_Avx(vcmpps, _CMP_EQ_OQ);
3813 }
3814 else {
3815 ✗ CmpOp_Avx(vcmpps, _CMP_EQ_OQ);
3816 }
3817 }
3818 ✗ else if (iter.op == opNotEq) { // avs+
3819 if (processSingle) {
3820 ✗ CmpOp_Single_Avx(vcmpps, _CMP_NEQ_OQ);
3821 }
3822 else {
3823 ✗ CmpOp_Avx(vcmpps, _CMP_NEQ_OQ);
3824 }
3825 }
3826 ✗ else if (iter.op == opLE) { // a <= b -> b >= (ge,nlt) a
3827 if (processSingle) {
3828 ✗ CmpOp_Single_Avx(vcmpps, _CMP_GE_OS); // cmpnltps
3829 }
3830 else {
3831 ✗ CmpOp_Avx(vcmpps, _CMP_GE_OS) // cmpnltps
3832 }
3833 }
3834 ✗ else if (iter.op == opGE) { // a >= b -> b <= (le) a
3835 if (processSingle) {
3836 ✗ CmpOp_Single_Avx(vcmpps, _CMP_LE_OS) // cmpleps
3837 }
3838 else {
3839 ✗ CmpOp_Avx(vcmpps, _CMP_LE_OS) // cmpleps
3840 }
3841 }
3842 ✗ else if (iter.op == opTernary) {
3843 if (processSingle) {
3844 ✗ auto t1 = stack1.back();
3845 ✗ stack1.pop_back();
3846 ✗ auto t2 = stack1.back();
3847 ✗ stack1.pop_back();
3848 ✗ auto t3 = stack1.back();
3849 ✗ stack1.pop_back();
3850 ✗ YmmReg r1;
3851 ✗ vxorps(r1, r1, r1);
3852 ✗ vcmpps(r1, r1, t3, _CMP_LT_OQ); // cmpltps -> vcmpps ... _CMP_LT_OQ
3853 ✗ vandps(t2, t2, r1);
3854 ✗ vandnps(r1, r1, t1);
3855 ✗ vorps(r1, r1, t2);
3856 ✗ stack1.push_back(r1);
3857 }
3858 else {
3859 ✗ auto t1 = stack.back();
3860 ✗ stack.pop_back();
3861 ✗ auto t2 = stack.back();
3862 ✗ stack.pop_back();
3863 ✗ auto t3 = stack.back();
3864 ✗ stack.pop_back();
3865 ✗ YmmReg r1, r2;
3866 ✗ vxorps(r1, r1, r1);
3867 ✗ vxorps(r2, r2, r2);
3868 ✗ vcmpps(r1, r1, t3.first, _CMP_LT_OQ); // cmpltps -> vcmpps ... _CMP_LT_OQ
3869 ✗ vcmpps(r2, r2, t3.second, _CMP_LT_OQ);
3870 ✗ vandps(t2.first, t2.first, r1);
3871 ✗ vandps(t2.second, t2.second, r2);
3872 ✗ vandnps(r1, r1, t1.first);
3873 ✗ vandnps(r2, r2, t1.second);
3874 ✗ vorps(r1, r1, t2.first);
3875 ✗ vorps(r2, r2, t2.second);
3876 ✗ stack.push_back(std::make_pair(r1, r2));
3877 }
3878 }
3879 ✗ else if (iter.op == opExp) {
3880 if (processSingle) {
3881 ✗ auto &t1 = stack1.back();
3882 ✗ EXP_PS_AVX(t1);
3883 }
3884 else {
3885 ✗ auto &t1 = stack.back();
3886 ✗ EXP_PS_AVX(t1.first);
3887 ✗ EXP_PS_AVX(t1.second);
3888 }
3889 }
3890 ✗ else if (iter.op == opLog) {
3891 if (processSingle) {
3892 ✗ auto &t1 = stack1.back();
3893 ✗ LOG_PS_AVX(t1);
3894 } else {
3895 ✗ auto &t1 = stack.back();
3896 ✗ LOG_PS_AVX(t1.first);
3897 ✗ LOG_PS_AVX(t1.second);
3898 }
3899 }
3900 ✗ else if (iter.op == opPow) {
3901 if (processSingle) {
3902 ✗ auto t1 = stack1.back();
3903 ✗ stack1.pop_back();
3904 ✗ auto &t2 = stack1.back();
3905 ✗ LOG_PS_AVX(t2);
3906 ✗ vmulps(t2, t2, t1);
3907 ✗ EXP_PS_AVX(t2);
3908 } else {
3909 ✗ auto t1 = stack.back();
3910 ✗ stack.pop_back();
3911 ✗ auto &t2 = stack.back();
3912 ✗ LOG_PS_AVX(t2.first);
3913 ✗ vmulps(t2.first, t2.first, t1.first);
3914 ✗ EXP_PS_AVX(t2.first);
3915 ✗ LOG_PS_AVX(t2.second);
3916 ✗ vmulps(t2.second, t2.second, t1.second);
3917 ✗ EXP_PS_AVX(t2.second);
3918 }
3919 }
3920 ✗ else if (iter.op == opSin) {
3921 if (processSingle) {
3922 ✗ auto& _t1 = stack1.back();
3923 ✗ SINCOS_PS_AVX(true, _t1, _t1);
3924 }
3925 else {
3926 ✗ auto& _t1 = stack.back();
3927 ✗ SINCOS_PS_AVX(true, _t1.first, _t1.first);
3928 ✗ SINCOS_PS_AVX(true, _t1.second, _t1.second);
3929 }
3930 }
3931 ✗ else if (iter.op == opCos) {
3932 if (processSingle) {
3933 ✗ auto& _t1 = stack1.back();
3934 ✗ SINCOS_PS_AVX(false, _t1, _t1);
3935 }
3936 else {
3937 ✗ auto& _t1 = stack.back();
3938 ✗ SINCOS_PS_AVX(false, _t1.first, _t1.first);
3939 ✗ SINCOS_PS_AVX(false, _t1.second, _t1.second);
3940 }
3941 }
3942 ✗ else if (iter.op == opTan) {
3943 if (processSingle) {
3944 ✗ auto& t1 = stack1.back();
3945 ✗ TAN_PS_AVX(t1);
3946 }
3947 else {
3948 ✗ auto& t1 = stack.back();
3949 ✗ TAN_PS_AVX(t1.first);
3950 ✗ TAN_PS_AVX(t1.second);
3951 }
3952 }
3953 ✗ else if (iter.op == opAtan2) {
3954 if (processSingle) {
3955 ✗ auto t1 = stack1.back();
3956 ✗ stack1.pop_back();
3957 ✗ auto &t2 = stack1.back();
3958 ✗ ATAN2_PS_AVX(t2, t1);
3959 } else {
3960 ✗ auto t1 = stack.back();
3961 ✗ stack.pop_back();
3962 ✗ auto &t2 = stack.back();
3963 ✗ ATAN2_PS_AVX(t2.first, t1.first);
3964 ✗ ATAN2_PS_AVX(t2.second, t1.second);
3965 }
3966 }
3967 ✗ else if (iter.op == opClip) {
3968 // clip(a, low, high) = min(max(a, low),high)
3969 if (processSingle) {
3970 ✗ auto t1 = stack1.back();
3971 ✗ stack1.pop_back();
3972 ✗ auto t2 = stack1.back();
3973 ✗ stack1.pop_back();
3974 ✗ auto &t3 = stack1.back();
3975 ✗ vmaxps(t3, t3, t2);
3976 ✗ vminps(t3, t3, t1);
3977 }
3978 else {
3979 ✗ auto t1 = stack.back();
3980 ✗ stack.pop_back();
3981 ✗ auto t2 = stack.back();
3982 ✗ stack.pop_back();
3983 ✗ auto &t3 = stack.back();
3984 ✗ vmaxps(t3.first, t3.first, t2.first);
3985 ✗ vminps(t3.first, t3.first, t1.first);
3986 ✗ vmaxps(t3.second, t3.second, t2.second);
3987 ✗ vminps(t3.second, t3.second, t1.second);
3988 }
3989 }
3990 ✗ else if (iter.op == opRound || iter.op == opFloor || iter.op == opCeil || iter.op == opTrunc) {
3991 ✗ const int rounder_flag =
3992 ✗ (iter.op == opRound) ? (FROUND_TO_NEAREST_INT | FROUND_NO_EXC) :
3993 ✗ (iter.op == opFloor) ? (FROUND_TO_NEG_INF | FROUND_NO_EXC) :
3994 ✗ (iter.op == opCeil) ? (FROUND_TO_POS_INF | FROUND_NO_EXC) :
3995 (FROUND_TO_ZERO | FROUND_NO_EXC); // opTrunc
3996 if (processSingle) {
3997 ✗ auto& t1 = stack1.back();
3998 ✗ vroundps(t1, t1, rounder_flag);
3999 }
4000 else {
4001 ✗ auto& t1 = stack.back();
4002 ✗ vroundps(t1.first, t1.first, rounder_flag);
4003 ✗ vroundps(t1.second, t1.second, rounder_flag);
4004 }
4005 }
4006 }
4007 12 }
4008 /*
4009 In brief:
4010 jitasm was modded to accept avx_epilog_=true for code generation
4011
4012 Why: couldn't use vzeroupper because prolog/epilog was saving all xmm6:xmm15 registers even if they were not used at all
4013 Why2: movaps was generated instead of vmovaps for prolog/epilog
4014 Why3: internal register reordering/saving was non-vex encoded
4015 All these issues resulted in AVX->SSE2 penalty
4016
4017 From MSDN:
4018 XMM6:XMM15, YMM6:YMM15 rules for x64:
4019 Nonvolatile (XMM), Volatile (upper half of YMM)
4020 XMM6:XMM15 Must be preserved as needed by callee.
4021 YMM registers must be preserved as needed by caller. (they do not need to be preserved)
4022
4023 Problem:
4024 - Jitasm saves xmm6..xmm15 when vzeroupper is used,even if only an xmm0 is used (Why?)
4025 No problem (looking at the disassembly list):
4026 - when there is no vzeroupper, then the xmm6:xmm11 is properly saved/restored in prolog/epilog but only
4027 if ymm6:ymm11 (in this example) is used. If no register is used over xmm6/ymm6 then xmm registers are not saved at all.
4028 - question: does it have any penalty when movaps is used w/o vzeroupper?
4029 The epilog generates movaps
4030 movaps xmm11,xmmword ptr [rbx-10h]
4031
4032 0000000002430000 push rbp
4033 0000000002430001 mov rbp,rsp
4034 0000000002430004 push rbx
4035 0000000002430005 lea rbx,[rsp-8]
4036 000000000243000A sub rsp,0A8h
4037 0000000002430011 movaps xmmword ptr [rbx-0A0h],xmm6
4038 0000000002430018 movaps xmmword ptr [rbx-90h],xmm7
4039 000000000243001F movaps xmmword ptr [rbx-80h],xmm8
4040 0000000002430024 movaps xmmword ptr [rbx-70h],xmm9
4041 0000000002430029 movaps xmmword ptr [rbx-60h],xmm10
4042 000000000243002E movaps xmmword ptr [rbx-50h],xmm11
4043 0000000002430033 movaps xmmword ptr [rbx-40h],xmm12
4044 0000000002430038 movaps xmmword ptr [rbx-30h],xmm13
4045 000000000243003D movaps xmmword ptr [rbx-20h],xmm14
4046 0000000002430042 movaps xmmword ptr [rbx-10h],xmm15
4047 -- end of jitasm generated prolog
4048
4049 -- PF AVX+: passing avx_epilog_ = true for codegen, vmovaps is generated instead of movaps
4050 0000000001E60010 vmovaps xmmword ptr [rbx-60h],xmm6
4051 0000000001E60015 vmovaps xmmword ptr [rbx-50h],xmm7
4052 0000000001E6001A vmovaps xmmword ptr [rbx-40h],xmm8
4053 0000000001E6001F vmovaps xmmword ptr [rbx-30h],xmm9
4054 0000000001E60024 vmovaps xmmword ptr [rbx-20h],xmm10
4055 0000000001E60029 vmovaps xmmword ptr [rbx-10h],xmm11
4056
4057 // PF comment: user's code like this:
4058 YmmReg zero;
4059 vpxor(zero, zero, zero);
4060 Reg constptr;
4061 mov(constptr, (uintptr_t)logexpconst_avx);
4062 vzeroupper();
4063 And the generated instructions:
4064 0000000002430047 vpxor ymm0,ymm0,ymm0
4065 000000000243004B mov rax,7FECCBD15C0h
4066 0000000002430055 vzeroupper
4067 Note: Don't use vzeroupper manually. When vzeroupper is issued manually, jitasm is not too generous: marks all xmm6:xmm15 registers as used
4068 and epilog and prolog will save all of them, even if none of those xmm/ymm registers are used in the code.
4069 Modded jitasm: pass avx_epilog_ = true for codegen, it will issue vzeroupper automatically (and has other benefits)
4070
4071 -- start of jitasm generated epilog (old)
4072 0000000002430058 movaps xmm15,xmmword ptr [rbx-10h]
4073 000000000243005D movaps xmm14,xmmword ptr [rbx-20h]
4074 0000000002430062 movaps xmm13,xmmword ptr [rbx-30h]
4075 0000000002430067 movaps xmm12,xmmword ptr [rbx-40h]
4076 000000000243006C movaps xmm11,xmmword ptr [rbx-50h]
4077 0000000002430071 movaps xmm10,xmmword ptr [rbx-60h]
4078 0000000002430076 movaps xmm9,xmmword ptr [rbx-70h]
4079 000000000243007B movaps xmm8,xmmword ptr [rbx-80h]
4080 0000000002430080 movaps xmm7,xmmword ptr [rbx-90h]
4081 0000000002430087 movaps xmm6,xmmword ptr [rbx-0A0h]
4082 000000000243008E add rsp,0A8h
4083 0000000002430095 pop rbx
4084 0000000002430096 pop rbp
4085 0000000002430097 ret
4086 -- end of jitasm generated epilog (old)
4087
4088 PF: modded jitasm (calling codegen with avx_epilog_ = true) generates vmovaps instead of movaps and and automatic vzeroupper before the ret instruction
4089 generated epilog example (new):
4090 0000000001E70613 vmovaps xmm11,xmmword ptr [rbx-10h]
4091 0000000001E70618 vmovaps xmm10,xmmword ptr [rbx-20h]
4092 0000000001E7061D vmovaps xmm9,xmmword ptr [rbx-30h]
4093 0000000001E70622 vmovaps xmm8,xmmword ptr [rbx-40h]
4094 0000000001E70627 vmovaps xmm7,xmmword ptr [rbx-50h]
4095 0000000001E7062C vmovaps xmm6,xmmword ptr [rbx-60h]
4096 0000000001E70631 add rsp,68h
4097 0000000001E70635 pop rdi
4098 0000000001E70636 pop rsi
4099 0000000001E70637 pop rbx
4100 0000000001E70638 pop rbp
4101 0000000001E70639 vzeroupper
4102 0000000001E7063C ret
4103
4104 */
4105 6 void main(Reg regptrs, Reg regoffs, Reg niter, Reg SpatialY)
4106 {
4107
1/2
✓ Branch 2 → 3 taken 6 times.
✗ Branch 2 → 10102 not taken.
 6 YmmReg zero;
4108
1/2
✓ Branch 3 → 4 taken 6 times.
✗ Branch 3 → 10102 not taken.
 6 vpxor(zero, zero, zero);
4109
1/2
✓ Branch 4 → 5 taken 6 times.
✗ Branch 4 → 10102 not taken.
 6 Reg constptr;
4110
2/4
✓ Branch 5 → 6 taken 6 times.
✗ Branch 5 → 10068 not taken.
✓ Branch 6 → 7 taken 6 times.
✗ Branch 6 → 10068 not taken.
 6 mov(constptr, (uintptr_t)logexpconst_avx);
4111
4112
2/4
✓ Branch 9 → 10 taken 6 times.
✗ Branch 9 → 10071 not taken.
✓ Branch 10 → 11 taken 6 times.
✗ Branch 10 → 10069 not taken.
 12 L("wloop");
4113
2/4
✓ Branch 13 → 14 taken 6 times.
✗ Branch 13 → 10075 not taken.
✓ Branch 14 → 15 taken 6 times.
✗ Branch 14 → 10075 not taken.
 6 cmp(niter, 0); // while(niter>0)
4114
2/4
✓ Branch 17 → 18 taken 6 times.
✗ Branch 17 → 10078 not taken.
✓ Branch 18 → 19 taken 6 times.
✗ Branch 18 → 10076 not taken.
 12 je("wend");
4115
2/4
✓ Branch 21 → 22 taken 6 times.
✗ Branch 21 → 10082 not taken.
✓ Branch 22 → 23 taken 6 times.
✗ Branch 22 → 10082 not taken.
 6 sub(niter, 1);
4116
4117 // process two sets, no partial input masking
4118
1/2
✗ Branch 23 → 24 not taken.
✓ Branch 23 → 1558 taken 6 times.
 6 if(singleMode)
4119 processingLoop<true, false>(regptrs, zero, constptr, SpatialY);
4120 else
4121 processingLoop<false, false>(regptrs, zero, constptr, SpatialY);
4122
4123 // increase read and write pointers by 16 pixels
4124 6 const int EXTRA = 2; // output pointer, xcounter
4125 if constexpr(sizeof(void *) == 8) {
4126 // x64: two 8 byte pointers in an xmm
4127 6 int numIter = (numInputs + EXTRA + 1) / 2;
4128
4129
2/2
✓ Branch 4254 → 4241 taken 12 times.
✓ Branch 4254 → 4255 taken 6 times.
 18 for (int i = 0; i < numIter; i++) {
4130
2/4
✓ Branch 4241 → 4242 taken 12 times.
✗ Branch 4241 → 10089 not taken.
✓ Branch 4242 → 4243 taken 12 times.
✗ Branch 4242 → 10089 not taken.
 12 XmmReg r1, r2;
4131
2/4
✓ Branch 4244 → 4245 taken 12 times.
✗ Branch 4244 → 10083 not taken.
✓ Branch 4245 → 4246 taken 12 times.
✗ Branch 4245 → 10083 not taken.
 12 vmovdqu(r1, xmmword_ptr[regptrs + 16 * i]);
4132
2/4
✓ Branch 4247 → 4248 taken 12 times.
✗ Branch 4247 → 10085 not taken.
✓ Branch 4248 → 4249 taken 12 times.
✗ Branch 4248 → 10085 not taken.
 12 vmovdqu(r2, xmmword_ptr[regoffs + 16 * i]);
4133
1/2
✓ Branch 4249 → 4250 taken 12 times.
✗ Branch 4249 → 10089 not taken.
 12 vpaddq(r1, r1, r2); // pointers are 64 bits
4134
2/4
✓ Branch 4251 → 4252 taken 12 times.
✗ Branch 4251 → 10087 not taken.
✓ Branch 4252 → 4253 taken 12 times.
✗ Branch 4252 → 10087 not taken.
 12 vmovdqu(xmmword_ptr[regptrs + 16 * i], r1);
4135 }
4136 }
4137 else {
4138 // x86: four 4 byte pointers in an xmm
4139 int numIter = (numInputs + EXTRA + 3) / 4;
4140 for (int i = 0; i < numIter; i++) {
4141 XmmReg r1, r2;
4142 vmovdqu(r1, xmmword_ptr[regptrs + 16 * i]);
4143 vmovdqu(r2, xmmword_ptr[regoffs + 16 * i]);
4144 vpaddd(r1, r1, r2); // pointers are 32 bits
4145 vmovdqu(xmmword_ptr[regptrs + 16 * i], r1);
4146 }
4147 }
4148
4149
2/4
✓ Branch 4257 → 4258 taken 6 times.
✗ Branch 4257 → 10092 not taken.
✓ Branch 4258 → 4259 taken 6 times.
✗ Branch 4258 → 10090 not taken.
 12 jmp("wloop");
4150
2/4
✓ Branch 4263 → 4264 taken 6 times.
✗ Branch 4263 → 10098 not taken.
✓ Branch 4264 → 4265 taken 6 times.
✗ Branch 4264 → 10096 not taken.
 12 L("wend");
4151
4152
1/2
✗ Branch 4267 → 4268 not taken.
✓ Branch 4267 → 4269 taken 6 times.
 6 int nrestpixels = planewidth & (singleMode ? 7 : 15);
4153
1/2
✗ Branch 4270 → 4271 not taken.
✓ Branch 4270 → 6975 taken 6 times.
 6 if(nrestpixels > 8) // dual process with masking
4154 processingLoop<false, true>(regptrs, zero, constptr, SpatialY);
4155
1/2
✗ Branch 6975 → 6976 not taken.
✓ Branch 6975 → 8510 taken 6 times.
 6 else if (nrestpixels == 8) // single process, no masking
4156 processingLoop<true, false>(regptrs, zero, constptr, SpatialY);
4157
1/2
✓ Branch 8510 → 8511 taken 6 times.
✗ Branch 8510 → 10067 not taken.
 6 else if (nrestpixels > 0) // single process, masking
4158 processingLoop<true, true>(regptrs, zero, constptr, SpatialY);
4159 // bug in jitasm?
4160 // on x64, when this is here, debug throws an assert, that a register save/load has an
4161 // operand size 8 bit, instead of 128 (XMM) or 256 (YMM)
4162 // vzeroupper(); // don't use it directly. Generate code with avx_epilog_=true
4163 6 }
4164 };
4165
4166 #endif
4167
4168
4169 /********************************************************************
4170 ***** Declare index of new filters for Avisynth's filter engine *****
4171 ********************************************************************/
4172
4173 extern const AVSFunction Exprfilter_filters[] = {
4174 { "Expr", BUILTIN_FUNC_PREFIX, "c+s+[format]s[optAvx2]b[optSingleMode]b[optSSE2]b[scale_inputs]s[clamp_float]b[clamp_float_UV]b[lut]i[optVectorC]b", Exprfilter::Create },
4175 { 0 }
4176 };
4177
4178
4179 ✗ AVSValue __cdecl Exprfilter::Create(AVSValue args, void* , IScriptEnvironment* env) {
4180
4181 ✗ std::vector<PClip> children;
4182 ✗ std::vector<std::string> expressions;
4183 int next_paramindex;
4184
4185 // one or more clips
4186 ✗ if (args[0].IsArray() && args[0][0].IsClip()) { // c+s+ case
4187 ✗ children.resize(args[0].ArraySize());
4188
4189 ✗ for (int i = 0; i < (int)children.size(); ++i) // Copy all
4190 ✗ children[i] = args[0][i].AsClip();
4191
4192 ✗ next_paramindex = 1;
4193 }
4194 ✗ else if (args[1].IsArray() && args[1][0].IsClip()) { // cc+s+ case
4195 ✗ children.resize(1 + args[1].ArraySize());
4196
4197 ✗ children[0] = args[0].AsClip(); // Copy 1st
4198 ✗ for (int i = 1; i < (int)children.size(); ++i) // Copy rest
4199 ✗ children[i] = args[1][i - 1].AsClip();
4200
4201 ✗ next_paramindex = 2;
4202 }
4203 ✗ else if (args[1].IsClip()) { //cc case
4204 ✗ children.resize(2);
4205
4206 ✗ children[0] = args[0].AsClip();
4207 ✗ children[1] = args[1].AsClip();
4208
4209 ✗ next_paramindex = 2;
4210 }
4211 ✗ else if (args[0].IsClip()) { // single clip, cs+ case
4212 ✗ children.resize(1);
4213 ✗ children[0] = args[0].AsClip();
4214
4215 ✗ next_paramindex = 1;
4216 }
4217 else {
4218 ✗ env->ThrowError("Expr: Invalid parameter type");
4219 }
4220
4221 // one or more expressions: s+
4222 ✗ if (args[next_paramindex].Defined()) {
4223 ✗ AVSValue exprarg = args[next_paramindex++];
4224 ✗ if (exprarg.IsArray()) {
4225 ✗ int nexpr = exprarg.ArraySize();
4226 ✗ expressions.resize(nexpr);
4227 ✗ for (int i = 0; i < nexpr; i++)
4228 ✗ expressions[i] = exprarg[i].AsString();
4229 }
4230 ✗ else if (exprarg.IsString()) {
4231 ✗ expressions.resize(1);
4232 ✗ expressions[0] = exprarg.AsString();
4233 }
4234 else {
4235 ✗ env->ThrowError("Expr: Invalid parameter type for expression string");
4236 }
4237 ✗ }
4238
4239 // optional named argument: format
4240 ✗ const char *newformat = nullptr;
4241 ✗ if (args[next_paramindex].Defined()) {
4242 // always string
4243 ✗ newformat = args[next_paramindex].AsString();
4244 }
4245 ✗ next_paramindex++;
4246
4247 #ifdef VS_TARGET_CPU_X86
4248 // test parameter for avx2-less mode even with avx2 available
4249 #ifdef TEST_AVX2_CODEGEN_IN_AVX
4250 bool optAvx2 = !!(env->GetCPUFlags() & CPUF_AVX);
4251 #else
4252 ✗ bool optAvx2 = !!(env->GetCPUFlags() & CPUF_AVX2);
4253 #endif
4254 ✗ bool optSSE2 = !!(env->GetCPUFlags() & CPUF_SSE2);
4255 #else
4256 bool optAvx2 = false;
4257 bool optSSE2 = false;
4258 #endif
4259
4260 ✗ if (args[next_paramindex].Defined()) {
4261 ✗ if (optAvx2) // disable only
4262 ✗ optAvx2 = args[next_paramindex].AsBool();
4263 }
4264 ✗ next_paramindex++;
4265
4266 ✗ bool optSingleMode = false;
4267 ✗ if (args[next_paramindex].Defined()) {
4268 ✗ optSingleMode = args[next_paramindex].AsBool();
4269 }
4270 ✗ next_paramindex++;
4271
4272 ✗ if (args[next_paramindex].Defined()) {
4273 ✗ if (optSSE2) // disable only
4274 ✗ optSSE2 = args[next_paramindex].AsBool();
4275 }
4276 ✗ next_paramindex++;
4277
4278 ✗ std::string scale_inputs = args[next_paramindex].Defined() ? args[next_paramindex].AsString("none") : "none";
4279 ✗ transform(scale_inputs.begin(), scale_inputs.end(), scale_inputs.begin(), ::tolower);
4280 ✗ next_paramindex++;
4281
4282 ✗ const bool clamp_float = args[next_paramindex].AsBool(false);
4283 ✗ next_paramindex++;
4284
4285 ✗ const bool clamp_float_UV = args[next_paramindex].AsBool(false);
4286 ✗ next_paramindex++;
4287
4288 // clamp_float clamp_float_uv -> clamp_float_i clamp range for Y clamp range for UV
4289 // false x 0 0..1 -0.5..+0.5
4290 // true false 1 0..1 -0.5..+0.5
4291 // true true 2 0..1 0..1
4292
4293 int clamp_float_i;
4294 ✗ if (clamp_float)
4295 ✗ clamp_float_i = clamp_float_UV ? 2 : 1;
4296 else
4297 ✗ clamp_float_i = 0;
4298
4299 ✗ const int lutmode = args[next_paramindex].AsInt(0); // 0, 1, 2
4300 ✗ next_paramindex++;
4301
4302 ✗ const bool optVectorC = args[next_paramindex].AsBool(true);
4303
4304 ✗ return new Exprfilter(children, expressions, newformat, optAvx2, optSingleMode, optSSE2, optVectorC, scale_inputs, clamp_float_i, lutmode, env);
4305
4306 ✗ }
4307
4308 // Base SIMD Processor interface
4309 class ISIMDProcessor {
4310 public:
4311 virtual void processVector(
4312 std::vector<const uint8_t*>&srcp,
4313 uint8_t*& dstp,
4314 int x, int y) = 0;
4315 32 virtual ~ISIMDProcessor() = default;
4316 };
4317
4318 /**
4319 * Custom aligned memory allocator for STL containers like std::vector
4320 *
4321 * This allocator ensures that the underlying data pointer returned by container.data()
4322 * is aligned to the specified alignment boundary, which is critical for:
4323 * - SIMD vector operations that require aligned memory access
4324 * - Cache-friendly data structures
4325 * - Hardware-specific memory alignment requirements
4326 *
4327 * Platform-specific implementation:
4328 * - Windows (MSVC, ClangCL): Uses _aligned_malloc/_aligned_free
4329 * - MinGW: Uses _aligned_malloc/_aligned_free
4330 * - POSIX systems (Linux, Unix, macOS): Uses aligned_alloc/free
4331 * - C++17 fallback: Uses std::aligned_alloc/free
4332 *
4333 * Usage:
4334 * std::vector<float, aligned_allocator<float, 32>> aligned_vector;
4335 */
4336 template <typename T, size_t Alignment>
4337 struct aligned_allocator {
4338 // Standard allocator typedefs
4339 typedef T value_type;
4340 typedef T* pointer;
4341 typedef const T* const_pointer;
4342 typedef T& reference;
4343 typedef const T& const_reference;
4344 typedef std::size_t size_type;
4345 typedef std::ptrdiff_t difference_type;
4346 // Rebind allocator to type U
4347 template <typename U>
4348 struct rebind {
4349 typedef aligned_allocator<U, Alignment> other;
4350 };
4351 40 aligned_allocator() noexcept {}
4352 template <typename U>
4353 aligned_allocator(const aligned_allocator<U, Alignment>&) noexcept {}
4354 112 T* allocate(std::size_t n) {
4355 #if defined(_MSC_VER) || (defined(__clang__) && defined(_MSC_VER))
4356 // Both MSVC and ClangCL should use _aligned_malloc
4357 void* ptr = _aligned_malloc(n * sizeof(T), Alignment);
4358 if (!ptr) throw std::bad_alloc();
4359 #elif defined(__MINGW32__) || defined(__MINGW64__)
4360 // MinGW/MinGW-w64 specific
4361 void* ptr = _aligned_malloc(n * sizeof(T), Alignment);
4362 if (!ptr) throw std::bad_alloc();
4363 #elif defined(__INTEL_COMPILER) || defined(__INTEL_LLVM_COMPILER) || defined(__APPLE__) || (defined(__GNUC__) && !defined(_WIN32) && !defined(__CYGWIN__))
4364 // POSIX-compliant systems: Intel compilers, macOS, GCC on non-Windows
4365 // aligned_alloc requires size to be a multiple of alignment
4366 112 size_t size = n * sizeof(T);
4367
2/2
✓ Branch 2 → 3 taken 52 times.
✓ Branch 2 → 4 taken 60 times.
 112 if (size % Alignment != 0) {
4368 52 size = (size / Alignment + 1) * Alignment;
4369 }
4370 112 void* ptr = aligned_alloc(Alignment, size);
4371
1/2
✗ Branch 4 → 5 not taken.
✓ Branch 4 → 8 taken 112 times.
 112 if (!ptr) throw std::bad_alloc();
4372 #else
4373 // Generic fallback for C++17 and later
4374 // std::aligned_alloc requires size to be a multiple of alignment
4375 size_t size = n * sizeof(T);
4376 if (size % Alignment != 0) {
4377 size = (size / Alignment + 1) * Alignment;
4378 }
4379 void* ptr = std::aligned_alloc(Alignment, size);
4380 if (!ptr) throw std::bad_alloc();
4381 #endif
4382 112 return static_cast<T*>(ptr);
4383 }
4384 112 void deallocate(T* p, std::size_t) noexcept {
4385 #if defined(_MSC_VER) || defined(__MINGW32__) || defined(__MINGW64__) || (defined(__clang__) && defined(_MSC_VER))
4386 _aligned_free(p);
4387 #else
4388 112 free(p);
4389 #endif
4390 112 }
4391 };
4392
4393 // convenient type def
4394 using aligned_float_vector = std::vector<float, aligned_allocator<float, 32>>;
4395
4396 template<int VectorSize>
4397 class SIMDProcessor : public ISIMDProcessor {
4398 int w, h;
4399 size_t maxStackSize;
4400
4401 // Define the aligned stack with custom allocator
4402 std::vector<aligned_float_vector> stack;
4403
4404 aligned_float_vector& variable_area;
4405 std::vector<float>& internal_vars;
4406 std::vector<int>& src_stride;
4407 std::vector<const uint8_t*>& srcp_orig;
4408 const ExprOp* vops;
4409 public:
4410 32 SIMDProcessor(int _w, int _h, size_t _maxStackSize,
4411 aligned_float_vector& var_area,
4412 std::vector<float>& int_vars, std::vector<int>& src_str, std::vector<const uint8_t*>& src_orig, const ExprOp* vops)
4413 32 : w(_w), h(_h), maxStackSize(_maxStackSize),
4414
8/16
SIMDProcessor<16>::SIMDProcessor(int, int, unsigned long, std::vector<float, aligned_allocator<float, 32ul> >&, std::vector<float, std::allocator<float> >&, std::vector<int, std::allocator<int> >&, std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, ExprOp const*):
✓ Branch 6 → 7 taken 8 times.
✗ Branch 6 → 13 not taken.
✓ Branch 7 → 8 taken 8 times.
✗ Branch 7 → 11 not taken.
SIMDProcessor<1>::SIMDProcessor(int, int, unsigned long, std::vector<float, aligned_allocator<float, 32ul> >&, std::vector<float, std::allocator<float> >&, std::vector<int, std::allocator<int> >&, std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, ExprOp const*):
✓ Branch 6 → 7 taken 8 times.
✗ Branch 6 → 13 not taken.
✓ Branch 7 → 8 taken 8 times.
✗ Branch 7 → 11 not taken.
SIMDProcessor<4>::SIMDProcessor(int, int, unsigned long, std::vector<float, aligned_allocator<float, 32ul> >&, std::vector<float, std::allocator<float> >&, std::vector<int, std::allocator<int> >&, std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, ExprOp const*):
✓ Branch 6 → 7 taken 8 times.
✗ Branch 6 → 13 not taken.
✓ Branch 7 → 8 taken 8 times.
✗ Branch 7 → 11 not taken.
SIMDProcessor<8>::SIMDProcessor(int, int, unsigned long, std::vector<float, aligned_allocator<float, 32ul> >&, std::vector<float, std::allocator<float> >&, std::vector<int, std::allocator<int> >&, std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, ExprOp const*):
✓ Branch 6 → 7 taken 8 times.
✗ Branch 6 → 13 not taken.
✓ Branch 7 → 8 taken 8 times.
✗ Branch 7 → 11 not taken.
 32 stack(_maxStackSize, aligned_float_vector(VectorSize)),
4415 32 variable_area(var_area), internal_vars(int_vars), src_stride(src_str), srcp_orig(src_orig), vops(vops) {
4416 32 }
4417
4418 int stackIndex = 0;
4419 alignas(32) float stacktop[VectorSize] = {};
4420
4421 // Broadcast a single value across the vector
4422 84 inline void push_and_broadcast(float val) {
4423 84 auto& current_stack = stack[stackIndex];
4424
4425 // First loop: copy stacktop to stack - help vectorization pattern
4426
6/8
SIMDProcessor<16>::push_and_broadcast(float):
✓ Branch 6 → 4 taken 560 times.
✓ Branch 6 → 7 taken 35 times.
SIMDProcessor<1>::push_and_broadcast(float):
✓ Branch 6 → 4 taken 39 times.
✓ Branch 6 → 7 taken 39 times.
SIMDProcessor<4>::push_and_broadcast(float):
✓ Branch 6 → 4 taken 40 times.
✓ Branch 6 → 7 taken 10 times.
SIMDProcessor<8>::push_and_broadcast(float):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
 723 for (int i = 0; i < VectorSize; ++i)
4427 639 current_stack[i] = stacktop[i];
4428
6/8
SIMDProcessor<16>::push_and_broadcast(float):
✓ Branch 9 → 8 taken 560 times.
✓ Branch 9 → 10 taken 35 times.
SIMDProcessor<1>::push_and_broadcast(float):
✓ Branch 9 → 8 taken 39 times.
✓ Branch 9 → 10 taken 39 times.
SIMDProcessor<4>::push_and_broadcast(float):
✓ Branch 9 → 8 taken 40 times.
✓ Branch 9 → 10 taken 10 times.
SIMDProcessor<8>::push_and_broadcast(float):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
 723 for (int i = 0; i < VectorSize; ++i)
4429 639 stacktop[i] = val;
4430 84 stackIndex++;
4431 84 }
4432
4433 125 inline void broadcast(float val) {
4434
6/8
SIMDProcessor<16>::broadcast(float):
✓ Branch 4 → 3 taken 704 times.
✓ Branch 4 → 5 taken 44 times.
SIMDProcessor<1>::broadcast(float):
✓ Branch 4 → 3 taken 66 times.
✓ Branch 4 → 5 taken 66 times.
SIMDProcessor<4>::broadcast(float):
✓ Branch 4 → 3 taken 60 times.
✓ Branch 4 → 5 taken 15 times.
SIMDProcessor<8>::broadcast(float):
✗ Branch 4 → 3 not taken.
✗ Branch 4 → 5 not taken.
 955 for (int i = 0; i < VectorSize; ++i)
4435 830 stacktop[i] = val;
4436 125 }
4437
4438 // Load spatial X coordinate
4439 66 inline void loadSpatialX(int x) {
4440 // Push current stacktop and fills x, x+1, x+2, x+3, ...
4441 66 auto& current_stack = stack[stackIndex];
4442
6/8
SIMDProcessor<16>::loadSpatialX(int):
✓ Branch 6 → 4 taken 416 times.
✓ Branch 6 → 7 taken 26 times.
SIMDProcessor<1>::loadSpatialX(int):
✓ Branch 6 → 4 taken 30 times.
✓ Branch 6 → 7 taken 30 times.
SIMDProcessor<4>::loadSpatialX(int):
✓ Branch 6 → 4 taken 40 times.
✓ Branch 6 → 7 taken 10 times.
SIMDProcessor<8>::loadSpatialX(int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
 552 for (int i = 0; i < VectorSize; ++i)
4443 486 current_stack[i] = stacktop[i];
4444
6/8
SIMDProcessor<16>::loadSpatialX(int):
✓ Branch 9 → 8 taken 416 times.
✓ Branch 9 → 10 taken 26 times.
SIMDProcessor<1>::loadSpatialX(int):
✓ Branch 9 → 8 taken 30 times.
✓ Branch 9 → 10 taken 30 times.
SIMDProcessor<4>::loadSpatialX(int):
✓ Branch 9 → 8 taken 40 times.
✓ Branch 9 → 10 taken 10 times.
SIMDProcessor<8>::loadSpatialX(int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
 552 for (int i = 0; i < VectorSize; ++i)
4445 486 stacktop[i] = static_cast<float>(x + i);
4446 66 stackIndex++;
4447 66 }
4448
4449 // Load vector from source from x
4450 template<typename T>
4451 93 inline void loadSource(const T* src, int x) {
4452 93 auto& current_stack = stack[stackIndex];
4453
10/24
void SIMDProcessor<16>::loadSource<float>(float const*, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<16>::loadSource<unsigned char>(unsigned char const*, int):
✓ Branch 6 → 4 taken 240 times.
✓ Branch 6 → 7 taken 15 times.
void SIMDProcessor<16>::loadSource<unsigned short>(unsigned short const*, int):
✓ Branch 6 → 4 taken 144 times.
✓ Branch 6 → 7 taken 9 times.
void SIMDProcessor<1>::loadSource<float>(float const*, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<1>::loadSource<unsigned char>(unsigned char const*, int):
✓ Branch 6 → 4 taken 45 times.
✓ Branch 6 → 7 taken 45 times.
void SIMDProcessor<1>::loadSource<unsigned short>(unsigned short const*, int):
✓ Branch 6 → 4 taken 9 times.
✓ Branch 6 → 7 taken 9 times.
void SIMDProcessor<4>::loadSource<float>(float const*, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<4>::loadSource<unsigned char>(unsigned char const*, int):
✓ Branch 6 → 4 taken 60 times.
✓ Branch 6 → 7 taken 15 times.
void SIMDProcessor<4>::loadSource<unsigned short>(unsigned short const*, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::loadSource<float>(float const*, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::loadSource<unsigned char>(unsigned char const*, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::loadSource<unsigned short>(unsigned short const*, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
 591 for (int i = 0; i < VectorSize; ++i)
4454 498 current_stack[i] = stacktop[i];
4455
10/24
void SIMDProcessor<16>::loadSource<float>(float const*, int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
void SIMDProcessor<16>::loadSource<unsigned char>(unsigned char const*, int):
✓ Branch 9 → 8 taken 240 times.
✓ Branch 9 → 10 taken 15 times.
void SIMDProcessor<16>::loadSource<unsigned short>(unsigned short const*, int):
✓ Branch 9 → 8 taken 144 times.
✓ Branch 9 → 10 taken 9 times.
void SIMDProcessor<1>::loadSource<float>(float const*, int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
void SIMDProcessor<1>::loadSource<unsigned char>(unsigned char const*, int):
✓ Branch 9 → 8 taken 45 times.
✓ Branch 9 → 10 taken 45 times.
void SIMDProcessor<1>::loadSource<unsigned short>(unsigned short const*, int):
✓ Branch 9 → 8 taken 9 times.
✓ Branch 9 → 10 taken 9 times.
void SIMDProcessor<4>::loadSource<float>(float const*, int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
void SIMDProcessor<4>::loadSource<unsigned char>(unsigned char const*, int):
✓ Branch 9 → 8 taken 60 times.
✓ Branch 9 → 10 taken 15 times.
void SIMDProcessor<4>::loadSource<unsigned short>(unsigned short const*, int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
void SIMDProcessor<8>::loadSource<float>(float const*, int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
void SIMDProcessor<8>::loadSource<unsigned char>(unsigned char const*, int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
void SIMDProcessor<8>::loadSource<unsigned short>(unsigned short const*, int):
✗ Branch 9 → 8 not taken.
✗ Branch 9 → 10 not taken.
 591 for (int i = 0; i < VectorSize; ++i)
4456 498 stacktop[i] = static_cast<float>(src[x + i]);
4457 93 stackIndex++;
4458 93 }
4459
4460 // Load from source with relative offset
4461 template<typename T>
4462 16 void loadRelSource(const T* src, int x, int y, int dx, int dy, int width, int height, int stride) {
4463 16 auto& current_stack = stack[stackIndex];
4464
4/24
void SIMDProcessor<16>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<16>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✓ Branch 6 → 4 taken 64 times.
✓ Branch 6 → 7 taken 4 times.
void SIMDProcessor<16>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<1>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<1>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✓ Branch 6 → 4 taken 12 times.
✓ Branch 6 → 7 taken 12 times.
void SIMDProcessor<1>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<4>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<4>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<4>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
 92 for (int i = 0; i < VectorSize; ++i)
4465 76 current_stack[i] = stacktop[i];
4466 // at the edges repeat, no mirror
4467 // stride is of byte pitch
4468 16 int newY = std::max(0, std::min(y + dy, height - 1)) - y;
4469 16 const uint8_t* rowPtr = reinterpret_cast<const uint8_t*>(src) + static_cast<intptr_t>(newY) * stride;
4470
4/24
void SIMDProcessor<16>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<16>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✓ Branch 13 → 10 taken 64 times.
✓ Branch 13 → 14 taken 4 times.
void SIMDProcessor<16>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<1>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<1>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✓ Branch 13 → 10 taken 12 times.
✓ Branch 13 → 14 taken 12 times.
void SIMDProcessor<1>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<4>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<4>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<4>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<8>::loadRelSource<float>(float const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<8>::loadRelSource<unsigned char>(unsigned char const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
void SIMDProcessor<8>::loadRelSource<unsigned short>(unsigned short const*, int, int, int, int, int, int, int):
✗ Branch 13 → 10 not taken.
✗ Branch 13 → 14 not taken.
 92 for (int i = 0; i < VectorSize; ++i) {
4471 76 int newX = std::max(0, std::min(x + dx + i, width - 1));
4472 76 stacktop[i] = static_cast<float>(reinterpret_cast<const T*>(rowPtr)[newX]);
4473 }
4474 16 stackIndex++;
4475 16 }
4476
4477 // Load from user variable area
4478 ✗ inline void loadVar(int index) {
4479 ✗ auto& current_stack = stack[stackIndex];
4480 ✗ for (int i = 0; i < VectorSize; ++i)
4481 ✗ current_stack[i] = stacktop[i];
4482 // Push current stacktop and broadcast x
4483 ✗ for (int i = 0; i < VectorSize; ++i)
4484 ✗ stacktop[i] = variable_area[index * MAX_C_VECT + i];
4485 ✗ stackIndex++;
4486 ✗ }
4487
4488 // Vectorized duplicate
4489 ✗ inline void dup(int offset) {
4490 ✗ auto& current_stack = stack[stackIndex];
4491 ✗ for (int i = 0; i < VectorSize; ++i)
4492 ✗ current_stack[i] = stacktop[i];
4493 ✗ auto& prev = stack[stackIndex - offset];
4494 ✗ for (int i = 0; i < VectorSize; ++i)
4495 ✗ stacktop[i] = prev[i];
4496 ✗ stackIndex++;
4497 ✗ }
4498
4499 // Vectorized swap
4500 ✗ inline void swap(int offset) {
4501 ✗ auto& prev = stack[stackIndex - offset];
4502 ✗ for (int i = 0; i < VectorSize; ++i)
4503 ✗ std::swap(stacktop[i], prev[i]);
4504 ✗ }
4505
4506 // Vectorized addition
4507 118 inline void add() {
4508 118 stackIndex--;
4509 118 auto& prev = stack[stackIndex];
4510
6/8
SIMDProcessor<16>::add():
✓ Branch 6 → 4 taken 464 times.
✓ Branch 6 → 7 taken 29 times.
SIMDProcessor<1>::add():
✓ Branch 6 → 4 taken 69 times.
✓ Branch 6 → 7 taken 69 times.
SIMDProcessor<4>::add():
✓ Branch 6 → 4 taken 80 times.
✓ Branch 6 → 7 taken 20 times.
SIMDProcessor<8>::add():
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
 731 for (int i = 0; i < VectorSize; ++i)
4511 613 stacktop[i] += prev[i];
4512 118 }
4513
4514 // Vectorized subtract
4515 16 inline void sub() {
4516 16 stackIndex--;
4517 16 auto& prev = stack[stackIndex];
4518
2/8
SIMDProcessor<16>::sub():
✓ Branch 6 → 4 taken 256 times.
✓ Branch 6 → 7 taken 16 times.
SIMDProcessor<1>::sub():
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
SIMDProcessor<4>::sub():
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
SIMDProcessor<8>::sub():
✗ Branch 6 → 4 not taken.
✗ Branch 6 → 7 not taken.
 272 for (int i = 0; i < VectorSize; ++i)
4519 256 stacktop[i] = prev[i] - stacktop[i];
4520 16 }
4521
4522 // Vectorized multiplication
4523 ✗ inline void multiply() {
4524 ✗ stackIndex--;
4525 ✗ auto& prev = stack[stackIndex];
4526 ✗ for (int i = 0; i < VectorSize; ++i)
4527 ✗ stacktop[i] *= prev[i];
4528 ✗ }
4529
4530 ✗ inline void divide() {
4531 ✗ stackIndex--;
4532 ✗ auto& prev = stack[stackIndex];
4533 ✗ for (int i = 0; i < VectorSize; ++i)
4534 ✗ stacktop[i] = prev[i] / stacktop[i];
4535 ✗ }
4536
4537 ✗ void fmod() {
4538 ✗ stackIndex--;
4539 ✗ auto& prev = stack[stackIndex];
4540 ✗ for (int i = 0; i < VectorSize; ++i)
4541 ✗ stacktop[i] = std::fmod(prev[i], stacktop[i]);
4542 ✗ }
4543
4544 ✗ inline void max() {
4545 ✗ stackIndex--;
4546 ✗ auto& prev = stack[stackIndex];
4547 ✗ for (int i = 0; i < VectorSize; ++i)
4548 ✗ stacktop[i] = std::max(prev[i], stacktop[i]);
4549 ✗ }
4550
4551 ✗ inline void min() {
4552 ✗ stackIndex--;
4553 ✗ auto& prev = stack[stackIndex];
4554 ✗ for (int i = 0; i < VectorSize; ++i)
4555 ✗ stacktop[i] = std::min(prev[i], stacktop[i]);
4556 ✗ }
4557
4558 ✗ void exp() {
4559 ✗ for (int i = 0; i < VectorSize; ++i)
4560 ✗ stacktop[i] = std::exp(stacktop[i]);
4561 ✗ }
4562
4563 ✗ void log() {
4564 ✗ for (int i = 0; i < VectorSize; ++i)
4565 ✗ stacktop[i] = std::log(stacktop[i]);
4566 ✗ }
4567
4568 ✗ void pow() {
4569 ✗ stackIndex--;
4570 ✗ auto& prev = stack[stackIndex];
4571 ✗ for (int i = 0; i < VectorSize; ++i)
4572 ✗ stacktop[i] = std::pow(prev[i], stacktop[i]);
4573 ✗ }
4574
4575 // Vectorized clip
4576 ✗ inline void clip() {
4577 ✗ stackIndex -= 2;
4578 ✗ auto& prev = stack[stackIndex];
4579 ✗ auto& prev1 = stack[stackIndex + 1];
4580 ✗ for (int i = 0; i < VectorSize; ++i)
4581 ✗ stacktop[i] = std::max(std::min(prev[i], stacktop[i]), prev1[i]);
4582 ✗ }
4583
4584 // Vectorized round
4585 ✗ inline void round() {
4586 ✗ for (int i = 0; i < VectorSize; ++i)
4587 ✗ stacktop[i] = std::round(stacktop[i]);
4588 ✗ }
4589
4590 // Vectorized floor
4591 ✗ inline void floor() {
4592 ✗ for (int i = 0; i < VectorSize; ++i)
4593 ✗ stacktop[i] = std::floor(stacktop[i]);
4594 ✗ }
4595
4596 // Vectorized ceil
4597 ✗ void ceil() {
4598 ✗ for (int i = 0; i < VectorSize; ++i)
4599 ✗ stacktop[i] = std::ceil(stacktop[i]);
4600 ✗ }
4601
4602 // Vectorized trunc
4603 ✗ inline void trunc() {
4604 ✗ for (int i = 0; i < VectorSize; ++i)
4605 ✗ stacktop[i] = std::trunc(stacktop[i]);
4606 ✗ }
4607
4608 // Vectorized sqrt
4609 ✗ void sqrt() {
4610 ✗ for (int i = 0; i < VectorSize; ++i)
4611 ✗ stacktop[i] = std::sqrt(stacktop[i]);
4612 ✗ }
4613
4614 // Vectorized abs
4615 ✗ inline void abs() {
4616 ✗ for (int i = 0; i < VectorSize; ++i)
4617 ✗ stacktop[i] = std::abs(stacktop[i]);
4618 ✗ }
4619
4620 // Vectorized sgn
4621 ✗ inline void sgn() {
4622 ✗ for (int i = 0; i < VectorSize; ++i)
4623 ✗ stacktop[i] = stacktop[i] < 0 ? -1.0f : stacktop[i] > 0 ? 1.0f : 0.0f;
4624 ✗ }
4625
4626 // Vectorized sin
4627 ✗ void sin() {
4628 ✗ for (int i = 0; i < VectorSize; ++i)
4629 ✗ stacktop[i] = std::sin(stacktop[i]);
4630 ✗ }
4631
4632 // Vectorized cos
4633 ✗ void cos() {
4634 ✗ for (int i = 0; i < VectorSize; ++i)
4635 ✗ stacktop[i] = std::cos(stacktop[i]);
4636 ✗ }
4637
4638 // Vectorized tan
4639 ✗ void tan() {
4640 ✗ for (int i = 0; i < VectorSize; ++i)
4641 ✗ stacktop[i] = std::tan(stacktop[i]);
4642 // reference of JITAsm code test: fast_tanf(stacktop[i]);
4643 ✗ }
4644
4645 // Vectorized asin
4646 ✗ void asin() {
4647 ✗ for (int i = 0; i < VectorSize; ++i)
4648 ✗ stacktop[i] = std::asin(stacktop[i]);
4649 ✗ }
4650
4651 // Vectorized acos
4652 ✗ void acos() {
4653 ✗ for (int i = 0; i < VectorSize; ++i)
4654 ✗ stacktop[i] = std::acos(stacktop[i]);
4655 ✗ }
4656
4657 // Vectorized atan
4658 ✗ void atan() {
4659 ✗ for (int i = 0; i < VectorSize; ++i)
4660 ✗ stacktop[i] = std::atan(stacktop[i]);
4661 ✗ }
4662
4663 // Vectorized atan2
4664 ✗ void atan2() {
4665 ✗ stackIndex--;
4666 ✗ auto& prev = stack[stackIndex];
4667 ✗ for (int i = 0; i < VectorSize; ++i)
4668 ✗ stacktop[i] = std::atan2(prev[i], stacktop[i]); // y, x -> -Pi..+Pi
4669 ✗ }
4670
4671 // Vectorized greater than
4672 ✗ inline void gt() {
4673 ✗ stackIndex--;
4674 ✗ auto& prev = stack[stackIndex];
4675 ✗ for (int i = 0; i < VectorSize; ++i)
4676 ✗ stacktop[i] = (prev[i] > stacktop[i]) ? 1.0f : 0.0f;
4677 ✗ }
4678
4679 // Vectorized less than
4680 ✗ inline void lt() {
4681 ✗ stackIndex--;
4682 ✗ auto& prev = stack[stackIndex];
4683 ✗ for (int i = 0; i < VectorSize; ++i)
4684 ✗ stacktop[i] = (prev[i] < stacktop[i]) ? 1.0f : 0.0f;
4685 ✗ }
4686
4687 // Vectorized equal
4688 ✗ inline void eq() {
4689 ✗ stackIndex--;
4690 ✗ auto& prev = stack[stackIndex];
4691 ✗ for (int i = 0; i < VectorSize; ++i)
4692 ✗ stacktop[i] = (prev[i] == stacktop[i]) ? 1.0f : 0.0f; // consider with not 100% match, use epsilon
4693 ✗ }
4694
4695 // Vectorized not equal
4696 ✗ inline void notEq() {
4697 ✗ stackIndex--;
4698 ✗ auto& prev = stack[stackIndex];
4699 ✗ for (int i = 0; i < VectorSize; ++i)
4700 ✗ stacktop[i] = (prev[i] != stacktop[i]) ? 1.0f : 0.0f; // consider with not 100% match, use epsilon
4701 ✗ }
4702
4703 // Vectorized less than or equal
4704 ✗ inline void le() {
4705 ✗ stackIndex--;
4706 ✗ auto& prev = stack[stackIndex];
4707 ✗ for (int i = 0; i < VectorSize; ++i)
4708 ✗ stacktop[i] = (prev[i] <= stacktop[i]) ? 1.0f : 0.0f;
4709 ✗ }
4710
4711 // Vectorized greater than or equal
4712 ✗ inline void ge() {
4713 ✗ stackIndex--;
4714 ✗ auto& prev = stack[stackIndex];
4715 ✗ for (int i = 0; i < VectorSize; ++i)
4716 ✗ stacktop[i] = (prev[i] >= stacktop[i]) ? 1.0f : 0.0f;
4717 ✗ }
4718
4719 // Vectorized ternary
4720 ✗ inline void ternary() {
4721 ✗ stackIndex -= 2;
4722 ✗ auto& prev = stack[stackIndex];
4723 ✗ auto& prev1 = stack[stackIndex + 1];
4724 ✗ for (int i = 0; i < VectorSize; ++i)
4725 ✗ stacktop[i] = (prev[i] > 0) ? prev1[i] : stacktop[i];
4726 ✗ }
4727
4728 // Vectorized logical AND
4729 ✗ inline void logicalAnd() {
4730 ✗ stackIndex--;
4731 ✗ auto& prev = stack[stackIndex];
4732 ✗ for (int i = 0; i < VectorSize; ++i)
4733 ✗ stacktop[i] = (stacktop[i] > 0 && prev[i] > 0) ? 1.0f : 0.0f;
4734 ✗ }
4735
4736 // Vectorized logical OR
4737 ✗ inline void logicalOr() {
4738 ✗ stackIndex--;
4739 ✗ auto& prev = stack[stackIndex];
4740 ✗ for (int i = 0; i < VectorSize; ++i)
4741 ✗ stacktop[i] = (stacktop[i] > 0 || prev[i] > 0) ? 1.0f : 0.0f;
4742 ✗ }
4743
4744 // Vectorized logical XOR
4745 ✗ inline void logicalXor() {
4746 ✗ stackIndex--;
4747 ✗ auto& prev = stack[stackIndex];
4748 ✗ for (int i = 0; i < VectorSize; ++i)
4749 ✗ stacktop[i] = ((stacktop[i] > 0) != (prev[i] > 0)) ? 1.0f : 0.0f;
4750 ✗ }
4751
4752 // Vectorized logical NOT
4753 ✗ inline void logicalNot() {
4754 ✗ for (int i = 0; i < VectorSize; ++i)
4755 ✗ stacktop[i] = (stacktop[i] > 0) ? 0.0f : 1.0f;
4756 ✗ }
4757
4758 // Vectorized negation
4759 ✗ inline void negSign() {
4760 ✗ for (int i = 0; i < VectorSize; ++i)
4761 ✗ stacktop[i] = -stacktop[i];
4762 ✗ }
4763
4764 // Templatized store function
4765 template<typename T, int MaxValue>
4766 125 inline void store(T* dst, int x) {
4767
10/40
void SIMDProcessor<16>::store<unsigned char, 255>(unsigned char*, int):
✓ Branch 6 → 3 taken 560 times.
✓ Branch 6 → 7 taken 35 times.
void SIMDProcessor<16>::store<unsigned short, 1023>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<16>::store<unsigned short, 16383>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<16>::store<unsigned short, 4095>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<16>::store<unsigned short, 65535>(unsigned short*, int):
✓ Branch 6 → 3 taken 144 times.
✓ Branch 6 → 7 taken 9 times.
void SIMDProcessor<1>::store<unsigned char, 255>(unsigned char*, int):
✓ Branch 6 → 3 taken 57 times.
✓ Branch 6 → 7 taken 57 times.
void SIMDProcessor<1>::store<unsigned short, 1023>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<1>::store<unsigned short, 16383>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<1>::store<unsigned short, 4095>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<1>::store<unsigned short, 65535>(unsigned short*, int):
✓ Branch 6 → 3 taken 9 times.
✓ Branch 6 → 7 taken 9 times.
void SIMDProcessor<4>::store<unsigned char, 255>(unsigned char*, int):
✓ Branch 6 → 3 taken 60 times.
✓ Branch 6 → 7 taken 15 times.
void SIMDProcessor<4>::store<unsigned short, 1023>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<4>::store<unsigned short, 16383>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<4>::store<unsigned short, 4095>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<4>::store<unsigned short, 65535>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::store<unsigned char, 255>(unsigned char*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::store<unsigned short, 1023>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::store<unsigned short, 16383>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::store<unsigned short, 4095>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
void SIMDProcessor<8>::store<unsigned short, 65535>(unsigned short*, int):
✗ Branch 6 → 3 not taken.
✗ Branch 6 → 7 not taken.
 955 for (int i = 0; i < VectorSize; ++i)
4768 830 dst[x + i] = static_cast<T>(std::max(0.0f, std::min(stacktop[i], static_cast<float>(MaxValue))) + 0.5f);
4769 125 }
4770
4771 // Specialized store function for float
4772 ✗ inline void store(float* dst, int x) {
4773 ✗ for (int i = 0; i < VectorSize; ++i)
4774 ✗ dst[x + i] = stacktop[i];
4775 ✗ }
4776
4777 // Store variable
4778 ✗ inline void storeVar(int index) {
4779 ✗ for (int i = 0; i < VectorSize; ++i)
4780 ✗ variable_area[index * MAX_C_VECT + i] = stacktop[i];
4781 ✗ }
4782
4783 // Store variable and drop one element from the stack
4784 ✗ inline void storeVarAndDrop1(int index) {
4785 ✗ for (int i = 0; i < VectorSize; ++i)
4786 ✗ variable_area[index * MAX_C_VECT + i] = stacktop[i];
4787 ✗ stackIndex--;
4788 ✗ if (stackIndex >= 0) {
4789 ✗ auto& prev = stack[stackIndex];
4790 ✗ for (int i = 0; i < VectorSize; ++i)
4791 ✗ stacktop[i] = prev[i];
4792 }
4793 ✗ }
4794
4795 // Process a sequence of operations
4796 125 void processVector(std::vector<const uint8_t*>& srcp, uint8_t*& dstp, int x, int y) override {
4797
4798 // Reset stack
4799 125 stackIndex = 0;
4800 125 broadcast(0); // stacktop = 0
4801
4802 125 const ExprOp* vops_current = vops; // reset instruction pointer
4803
4804 while (true) {
4805 // Process instruction sequence
4806
24/240
SIMDProcessor<16>::processVector(std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, unsigned char*&, int, int):
✓ Branch 4 → 5 taken 15 times.
✓ Branch 4 → 8 taken 9 times.
✗ Branch 4 → 11 not taken.
✓ Branch 4 → 14 taken 4 times.
✗ Branch 4 → 18 not taken.
✗ Branch 4 → 22 not taken.
✓ Branch 4 → 26 taken 25 times.
✓ Branch 4 → 28 taken 26 times.
✓ Branch 4 → 30 taken 10 times.
✗ Branch 4 → 32 not taken.
✓ Branch 4 → 35 taken 35 times.
✗ Branch 4 → 37 not taken.
✗ Branch 4 → 39 not taken.
✗ Branch 4 → 41 not taken.
✓ Branch 4 → 43 taken 9 times.
✗ Branch 4 → 45 not taken.
✗ Branch 4 → 47 not taken.
✗ Branch 4 → 48 not taken.
✓ Branch 4 → 49 taken 29 times.
✓ Branch 4 → 50 taken 16 times.
✗ Branch 4 → 51 not taken.
✗ Branch 4 → 52 not taken.
✗ Branch 4 → 53 not taken.
✗ Branch 4 → 54 not taken.
✗ Branch 4 → 55 not taken.
✗ Branch 4 → 56 not taken.
✗ Branch 4 → 57 not taken.
✗ Branch 4 → 58 not taken.
✗ Branch 4 → 59 not taken.
✗ Branch 4 → 60 not taken.
✗ Branch 4 → 61 not taken.
✗ Branch 4 → 62 not taken.
✗ Branch 4 → 63 not taken.
✗ Branch 4 → 64 not taken.
✗ Branch 4 → 65 not taken.
✗ Branch 4 → 66 not taken.
✗ Branch 4 → 67 not taken.
✗ Branch 4 → 68 not taken.
✗ Branch 4 → 69 not taken.
✗ Branch 4 → 70 not taken.
✗ Branch 4 → 71 not taken.
✗ Branch 4 → 72 not taken.
✗ Branch 4 → 73 not taken.
✗ Branch 4 → 74 not taken.
✗ Branch 4 → 75 not taken.
✗ Branch 4 → 76 not taken.
✗ Branch 4 → 77 not taken.
✗ Branch 4 → 78 not taken.
✗ Branch 4 → 79 not taken.
✗ Branch 4 → 80 not taken.
✗ Branch 4 → 81 not taken.
✗ Branch 4 → 82 not taken.
✗ Branch 4 → 83 not taken.
✗ Branch 4 → 84 not taken.
✗ Branch 4 → 85 not taken.
✗ Branch 4 → 86 not taken.
✗ Branch 4 → 88 not taken.
✗ Branch 4 → 90 not taken.
✗ Branch 4 → 93 not taken.
✗ Branch 4 → 95 not taken.
SIMDProcessor<1>::processVector(std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, unsigned char*&, int, int):
✓ Branch 4 → 5 taken 45 times.
✓ Branch 4 → 8 taken 9 times.
✗ Branch 4 → 11 not taken.
✓ Branch 4 → 14 taken 12 times.
✗ Branch 4 → 18 not taken.
✗ Branch 4 → 22 not taken.
✓ Branch 4 → 26 taken 9 times.
✓ Branch 4 → 28 taken 30 times.
✓ Branch 4 → 30 taken 30 times.
✗ Branch 4 → 32 not taken.
✓ Branch 4 → 35 taken 57 times.
✗ Branch 4 → 37 not taken.
✗ Branch 4 → 39 not taken.
✗ Branch 4 → 41 not taken.
✓ Branch 4 → 43 taken 9 times.
✗ Branch 4 → 45 not taken.
✗ Branch 4 → 47 not taken.
✗ Branch 4 → 48 not taken.
✓ Branch 4 → 49 taken 69 times.
✗ Branch 4 → 50 not taken.
✗ Branch 4 → 51 not taken.
✗ Branch 4 → 52 not taken.
✗ Branch 4 → 53 not taken.
✗ Branch 4 → 54 not taken.
✗ Branch 4 → 55 not taken.
✗ Branch 4 → 56 not taken.
✗ Branch 4 → 57 not taken.
✗ Branch 4 → 58 not taken.
✗ Branch 4 → 59 not taken.
✗ Branch 4 → 60 not taken.
✗ Branch 4 → 61 not taken.
✗ Branch 4 → 62 not taken.
✗ Branch 4 → 63 not taken.
✗ Branch 4 → 64 not taken.
✗ Branch 4 → 65 not taken.
✗ Branch 4 → 66 not taken.
✗ Branch 4 → 67 not taken.
✗ Branch 4 → 68 not taken.
✗ Branch 4 → 69 not taken.
✗ Branch 4 → 70 not taken.
✗ Branch 4 → 71 not taken.
✗ Branch 4 → 72 not taken.
✗ Branch 4 → 73 not taken.
✗ Branch 4 → 74 not taken.
✗ Branch 4 → 75 not taken.
✗ Branch 4 → 76 not taken.
✗ Branch 4 → 77 not taken.
✗ Branch 4 → 78 not taken.
✗ Branch 4 → 79 not taken.
✗ Branch 4 → 80 not taken.
✗ Branch 4 → 81 not taken.
✗ Branch 4 → 82 not taken.
✗ Branch 4 → 83 not taken.
✗ Branch 4 → 84 not taken.
✗ Branch 4 → 85 not taken.
✗ Branch 4 → 86 not taken.
✗ Branch 4 → 88 not taken.
✗ Branch 4 → 90 not taken.
✗ Branch 4 → 93 not taken.
✗ Branch 4 → 95 not taken.
SIMDProcessor<4>::processVector(std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, unsigned char*&, int, int):
✓ Branch 4 → 5 taken 15 times.
✗ Branch 4 → 8 not taken.
✗ Branch 4 → 11 not taken.
✗ Branch 4 → 14 not taken.
✗ Branch 4 → 18 not taken.
✗ Branch 4 → 22 not taken.
✗ Branch 4 → 26 not taken.
✓ Branch 4 → 28 taken 10 times.
✓ Branch 4 → 30 taken 10 times.
✗ Branch 4 → 32 not taken.
✓ Branch 4 → 35 taken 15 times.
✗ Branch 4 → 37 not taken.
✗ Branch 4 → 39 not taken.
✗ Branch 4 → 41 not taken.
✗ Branch 4 → 43 not taken.
✗ Branch 4 → 45 not taken.
✗ Branch 4 → 47 not taken.
✗ Branch 4 → 48 not taken.
✓ Branch 4 → 49 taken 20 times.
✗ Branch 4 → 50 not taken.
✗ Branch 4 → 51 not taken.
✗ Branch 4 → 52 not taken.
✗ Branch 4 → 53 not taken.
✗ Branch 4 → 54 not taken.
✗ Branch 4 → 55 not taken.
✗ Branch 4 → 56 not taken.
✗ Branch 4 → 57 not taken.
✗ Branch 4 → 58 not taken.
✗ Branch 4 → 59 not taken.
✗ Branch 4 → 60 not taken.
✗ Branch 4 → 61 not taken.
✗ Branch 4 → 62 not taken.
✗ Branch 4 → 63 not taken.
✗ Branch 4 → 64 not taken.
✗ Branch 4 → 65 not taken.
✗ Branch 4 → 66 not taken.
✗ Branch 4 → 67 not taken.
✗ Branch 4 → 68 not taken.
✗ Branch 4 → 69 not taken.
✗ Branch 4 → 70 not taken.
✗ Branch 4 → 71 not taken.
✗ Branch 4 → 72 not taken.
✗ Branch 4 → 73 not taken.
✗ Branch 4 → 74 not taken.
✗ Branch 4 → 75 not taken.
✗ Branch 4 → 76 not taken.
✗ Branch 4 → 77 not taken.
✗ Branch 4 → 78 not taken.
✗ Branch 4 → 79 not taken.
✗ Branch 4 → 80 not taken.
✗ Branch 4 → 81 not taken.
✗ Branch 4 → 82 not taken.
✗ Branch 4 → 83 not taken.
✗ Branch 4 → 84 not taken.
✗ Branch 4 → 85 not taken.
✗ Branch 4 → 86 not taken.
✗ Branch 4 → 88 not taken.
✗ Branch 4 → 90 not taken.
✗ Branch 4 → 93 not taken.
✗ Branch 4 → 95 not taken.
SIMDProcessor<8>::processVector(std::vector<unsigned char const*, std::allocator<unsigned char const*> >&, unsigned char*&, int, int):
✗ Branch 4 → 5 not taken.
✗ Branch 4 → 8 not taken.
✗ Branch 4 → 11 not taken.
✗ Branch 4 → 14 not taken.
✗ Branch 4 → 18 not taken.
✗ Branch 4 → 22 not taken.
✗ Branch 4 → 26 not taken.
✗ Branch 4 → 28 not taken.
✗ Branch 4 → 30 not taken.
✗ Branch 4 → 32 not taken.
✗ Branch 4 → 35 not taken.
✗ Branch 4 → 37 not taken.
✗ Branch 4 → 39 not taken.
✗ Branch 4 → 41 not taken.
✗ Branch 4 → 43 not taken.
✗ Branch 4 → 45 not taken.
✗ Branch 4 → 47 not taken.
✗ Branch 4 → 48 not taken.
✗ Branch 4 → 49 not taken.
✗ Branch 4 → 50 not taken.
✗ Branch 4 → 51 not taken.
✗ Branch 4 → 52 not taken.
✗ Branch 4 → 53 not taken.
✗ Branch 4 → 54 not taken.
✗ Branch 4 → 55 not taken.
✗ Branch 4 → 56 not taken.
✗ Branch 4 → 57 not taken.
✗ Branch 4 → 58 not taken.
✗ Branch 4 → 59 not taken.
✗ Branch 4 → 60 not taken.
✗ Branch 4 → 61 not taken.
✗ Branch 4 → 62 not taken.
✗ Branch 4 → 63 not taken.
✗ Branch 4 → 64 not taken.
✗ Branch 4 → 65 not taken.
✗ Branch 4 → 66 not taken.
✗ Branch 4 → 67 not taken.
✗ Branch 4 → 68 not taken.
✗ Branch 4 → 69 not taken.
✗ Branch 4 → 70 not taken.
✗ Branch 4 → 71 not taken.
✗ Branch 4 → 72 not taken.
✗ Branch 4 → 73 not taken.
✗ Branch 4 → 74 not taken.
✗ Branch 4 → 75 not taken.
✗ Branch 4 → 76 not taken.
✗ Branch 4 → 77 not taken.
✗ Branch 4 → 78 not taken.
✗ Branch 4 → 79 not taken.
✗ Branch 4 → 80 not taken.
✗ Branch 4 → 81 not taken.
✗ Branch 4 → 82 not taken.
✗ Branch 4 → 83 not taken.
✗ Branch 4 → 84 not taken.
✗ Branch 4 → 85 not taken.
✗ Branch 4 → 86 not taken.
✗ Branch 4 → 88 not taken.
✗ Branch 4 → 90 not taken.
✗ Branch 4 → 93 not taken.
✗ Branch 4 → 95 not taken.
 518 switch (vops_current->op) {
4807 75 case opLoadSrc8:
4808 75 loadSource<uint8_t>(reinterpret_cast<const uint8_t*>(srcp[vops_current->e.ival]), x);
4809 75 break;
4810 18 case opLoadSrc16:
4811 18 loadSource<uint16_t>(reinterpret_cast<const uint16_t*>(srcp[vops_current->e.ival]), x);
4812 18 break;
4813 ✗ case opLoadSrcF32:
4814 ✗ loadSource<float>(reinterpret_cast<const float*>(srcp[vops_current->e.ival]), x);
4815 ✗ break;
4816 16 case opLoadRelSrc8:
4817 16 loadRelSource<uint8_t>(reinterpret_cast<const uint8_t*>(srcp[vops_current->e.ival]), x, y, vops_current->dx, vops_current->dy, w, h, src_stride[vops_current->e.ival]);
4818 16 break;
4819 ✗ case opLoadRelSrc16:
4820 ✗ loadRelSource<uint16_t>(reinterpret_cast<const uint16_t*>(srcp[vops_current->e.ival]), x, y, vops_current->dx, vops_current->dy, w, h, src_stride[vops_current->e.ival]);
4821 ✗ break;
4822 ✗ case opLoadRelSrcF32:
4823 ✗ loadRelSource<float>(reinterpret_cast<const float*>(srcp[vops_current->e.ival]), x, y, vops_current->dx, vops_current->dy, w, h, src_stride[vops_current->e.ival]);
4824 ✗ break;
4825 34 case opLoadConst:
4826 34 push_and_broadcast(vops_current->e.fval);
4827 34 break;
4828 66 case opLoadSpatialX:
4829 66 loadSpatialX(x);
4830 66 break;
4831 50 case opLoadSpatialY:
4832 50 push_and_broadcast(static_cast<float>(y));
4833 50 break;
4834 ✗ case opLoadInternalVar:
4835 ✗ push_and_broadcast(internal_vars[vops_current->e.ival]);
4836 ✗ break;
4837 107 case opStore8:
4838 107 store<uint8_t, 255>(reinterpret_cast<uint8_t*>(dstp), x);
4839 107 goto loopend;
4840 ✗ case opStore10:
4841 ✗ store<uint16_t, 1023>(reinterpret_cast<uint16_t*>(dstp), x);
4842 ✗ goto loopend;
4843 ✗ case opStore12:
4844 ✗ store<uint16_t, 4095>(reinterpret_cast<uint16_t*>(dstp), x);
4845 ✗ goto loopend;
4846 ✗ case opStore14:
4847 ✗ store<uint16_t, 16383>(reinterpret_cast<uint16_t*>(dstp), x);
4848 ✗ goto loopend;
4849 18 case opStore16:
4850 18 store<uint16_t, 65535>(reinterpret_cast<uint16_t*>(dstp), x);
4851 18 goto loopend;
4852 ✗ case opStoreF32:
4853 ✗ store(reinterpret_cast<float*>(dstp), x);
4854 ✗ goto loopend;
4855
4856 ✗ case opDup: dup(vops_current->e.ival); break;
4857 ✗ case opSwap: swap(vops_current->e.ival); break;
4858 118 case opAdd: add(); break;
4859 16 case opSub: sub(); break;
4860 ✗ case opMul: multiply(); break;
4861 ✗ case opDiv: divide(); break;
4862 ✗ case opMax: max(); break;
4863 ✗ case opMin: min(); break;
4864 ✗ case opSqrt: sqrt(); break;
4865 ✗ case opAbs: abs(); break;
4866 ✗ case opSgn: sgn(); break;
4867 ✗ case opFmod: fmod(); break;
4868 ✗ case opGt: gt(); break;
4869 ✗ case opLt: lt(); break;
4870 ✗ case opEq: eq(); break;
4871 ✗ case opNotEq: notEq(); break;
4872 ✗ case opLE: le(); break;
4873 ✗ case opGE: ge(); break;
4874 ✗ case opTernary: ternary(); break;
4875 ✗ case opAnd: logicalAnd(); break;
4876 ✗ case opOr: logicalOr(); break;
4877 ✗ case opXor: logicalXor(); break;
4878 ✗ case opNeg: logicalNot(); break;
4879 ✗ case opNegSign: negSign(); break;
4880 ✗ case opExp: exp(); break;
4881 ✗ case opLog: log(); break;
4882 ✗ case opPow: pow(); break;
4883 ✗ case opSin: sin(); break;
4884 ✗ case opCos: cos(); break;
4885 ✗ case opTan: tan(); break;
4886 ✗ case opAsin: asin(); break;
4887 ✗ case opAcos: acos(); break;
4888 ✗ case opAtan: atan(); break;
4889 ✗ case opAtan2: atan2(); break;
4890 ✗ case opClip: clip(); break;
4891 ✗ case opRound: round(); break;
4892 ✗ case opFloor: floor(); break;
4893 ✗ case opCeil: ceil(); break;
4894 ✗ case opTrunc: trunc(); break;
4895
4896 ✗ case opStoreVar:
4897 ✗ storeVar(vops_current->e.ival);
4898 ✗ break;
4899 ✗ case opLoadVar:
4900 ✗ loadVar(vops_current->e.ival);
4901 ✗ break;
4902 ✗ case opLoadFramePropVar:
4903 ✗ push_and_broadcast(internal_vars[INTERNAL_VAR_FRAMEPROP_VARIABLES_START + vops_current->e.ival]);
4904 ✗ break;
4905 ✗ case opStoreVarAndDrop1:
4906 ✗ storeVarAndDrop1(vops_current->e.ival);
4907 ✗ break;
4908 }
4909 393 vops_current++; // next opcode
4910 }
4911 // store makes the sequence end, wherever it was
4912 125 loopend:;
4913 125 }
4914 };
4915
4916
4917 // Factory to create appropriate SIMD processors
4918 class SIMDProcessorFactory {
4919 public:
4920 template<int MaxVectorSize>
4921 32 static std::unique_ptr<ISIMDProcessor> createProcessor(
4922 int w, int h, size_t maxStackSize,
4923 aligned_float_vector& var_area,
4924 std::vector<float>& int_vars, std::vector<int>& src_str, std::vector<const uint8_t*>& src_orig, const ExprOp* vops) {
4925 if constexpr (MaxVectorSize >= 16) {
4926
1/2
✓ Branch 2 → 3 taken 8 times.
✗ Branch 2 → 7 not taken.
 8 return std::make_unique<SIMDProcessor<16>>(w, h, maxStackSize, var_area, int_vars, src_str, src_orig, vops);
4927 }
4928 if constexpr (MaxVectorSize >= 8) {
4929
1/2
✓ Branch 2 → 3 taken 8 times.
✗ Branch 2 → 7 not taken.
 8 return std::make_unique<SIMDProcessor<8>>(w, h, maxStackSize, var_area, int_vars, src_str, src_orig, vops);
4930 }
4931 else if constexpr (MaxVectorSize >= 4) {
4932
1/2
✓ Branch 2 → 3 taken 8 times.
✗ Branch 2 → 7 not taken.
 8 return std::make_unique<SIMDProcessor<4>>(w, h, maxStackSize, var_area, int_vars, src_str, src_orig, vops);
4933 }
4934 else {
4935
1/2
✓ Branch 2 → 3 taken 8 times.
✗ Branch 2 → 7 not taken.
 8 return std::make_unique<SIMDProcessor<1>>(w, h, maxStackSize, var_area, int_vars, src_str, src_orig, vops);
4936 }
4937 }
4938 };
4939
4940
4941 template<int MaxVectorSize>
4942 8 void processFrameWithDynamicVectors(int plane, int w, int h, int pixels_per_iter, float framecount, float relative_time, int numInputs,
4943 uint8_t* &dstp, int dst_stride,
4944 std::vector<const uint8_t*>& srcp, std::vector<int>& src_stride, std::vector<intptr_t>& ptroffsets, std::vector<const uint8_t*>& srcp_orig, ExprData& d) {
4945
4946 8 const ExprOp* vops = d.ops[plane].data();
4947
1/2
✓ Branch 4 → 5 taken 8 times.
✗ Branch 4 → 72 not taken.
 8 aligned_float_vector variable_area(MAX_USER_VARIABLES * MAX_C_VECT); // for C, place for expr variables (each is a vector)
4948
4949
1/2
✓ Branch 7 → 8 taken 8 times.
✗ Branch 7 → 73 not taken.
 8 std::vector<float> internal_vars(INTERNAL_VARIABLES + MAX_FRAMEPROP_VARIABLES);
4950 // frame dependent internal variables
4951 8 internal_vars[INTERNAL_VAR_CURRENT_FRAME] = (float)framecount;
4952 8 internal_vars[INTERNAL_VAR_RELTIME] = (float)relative_time;
4953 // followed by dynamic frame properties
4954
1/2
✗ Branch 25 → 13 not taken.
✓ Branch 25 → 26 taken 8 times.
 16 for (auto& framePropToRead : d.frameprops[plane]) {
4955 ✗ int whereToPut = framePropToRead.var_index;
4956 ✗ internal_vars[INTERNAL_VAR_FRAMEPROP_VARIABLES_START + whereToPut] = framePropToRead.value;
4957 };
4958
4959 // Create processors dynamically based on MaxVectorSize
4960
1/2
✓ Branch 26 → 27 taken 8 times.
✗ Branch 26 → 84 not taken.
 8 std::unique_ptr<ISIMDProcessor> processor16 = MaxVectorSize >= 16 ?
4961 SIMDProcessorFactory::createProcessor<16>(w, h, d.maxStackSize, variable_area, internal_vars, src_stride, srcp_orig, vops) : nullptr;
4962
1/2
✓ Branch 27 → 28 taken 8 times.
✗ Branch 27 → 82 not taken.
 8 std::unique_ptr<ISIMDProcessor> processor8 = MaxVectorSize >= 8 ?
4963 SIMDProcessorFactory::createProcessor<8>(w, h, d.maxStackSize, variable_area, internal_vars, src_stride, srcp_orig, vops) : nullptr;
4964
1/2
✓ Branch 28 → 29 taken 8 times.
✗ Branch 28 → 80 not taken.
 8 std::unique_ptr<ISIMDProcessor> processor4 = MaxVectorSize >= 4 ?
4965 SIMDProcessorFactory::createProcessor<4>(w, h, d.maxStackSize, variable_area, internal_vars, src_stride, srcp_orig, vops) : nullptr;
4966
1/2
✓ Branch 29 → 30 taken 8 times.
✗ Branch 29 → 78 not taken.
 8 std::unique_ptr<ISIMDProcessor> processor1 = SIMDProcessorFactory::createProcessor<1>(w, h, d.maxStackSize, variable_area, internal_vars, src_stride, srcp_orig, vops);
4967
4968
2/2
✓ Branch 64 → 31 taken 29 times.
✓ Branch 64 → 65 taken 8 times.
 37 for (int y = 0; y < h; y++) {
4969 29 int x = 0;
4970
4971 // Conditionally process larger vector sizes
4972
1/2
✓ Branch 32 → 33 taken 29 times.
✗ Branch 32 → 38 not taken.
 29 if (MaxVectorSize >= 16 && processor16) {
4973
2/2
✓ Branch 37 → 34 taken 44 times.
✓ Branch 37 → 38 taken 29 times.
 73 for (; x + 16 <= w; x += 16) {
4974
1/2
✓ Branch 35 → 36 taken 44 times.
✗ Branch 35 → 76 not taken.
 44 processor16->processVector(srcp, dstp, x, y);
4975 }
4976 }
4977
4978
1/2
✓ Branch 39 → 40 taken 29 times.
✗ Branch 39 → 45 not taken.
 29 if (MaxVectorSize >= 8 && processor8) {
4979
1/2
✗ Branch 44 → 41 not taken.
✓ Branch 44 → 45 taken 29 times.
 29 for (; x + 8 <= w; x += 8) {
4980 ✗ processor8->processVector(srcp, dstp, x, y);
4981 }
4982 }
4983
4984
1/2
✓ Branch 46 → 47 taken 29 times.
✗ Branch 46 → 52 not taken.
 29 if (MaxVectorSize >= 4 && processor4) {
4985
2/2
✓ Branch 51 → 48 taken 15 times.
✓ Branch 51 → 52 taken 29 times.
 44 for (; x + 4 <= w; x += 4) {
4986
1/2
✓ Branch 49 → 50 taken 15 times.
✗ Branch 49 → 76 not taken.
 15 processor4->processVector(srcp, dstp, x, y);
4987 }
4988 }
4989
4990 // Always process remaining pixels
4991
2/2
✓ Branch 56 → 53 taken 66 times.
✓ Branch 56 → 57 taken 29 times.
 95 for (; x < w; x++) {
4992
1/2
✓ Branch 54 → 55 taken 66 times.
✗ Branch 54 → 76 not taken.
 66 processor1->processVector(srcp, dstp, x, y);
4993 }
4994
4995 // Update destination and source pointers for next row
4996 29 dstp += dst_stride;
4997
2/2
✓ Branch 57 → 58 taken 28 times.
✓ Branch 57 → 63 taken 1 time.
 29 if (d.lutmode == 0) {
4998
2/2
✓ Branch 62 → 59 taken 28 times.
✓ Branch 62 → 63 taken 28 times.
 56 for (int i = 0; i < numInputs; i++)
4999 28 srcp[i] += src_stride[i];
5000 }
5001 }
5002 8 }
5003
5004 33 void Exprfilter::processFrame(int plane, int w, int h, int pixels_per_iter, float framecount, float relative_time, int numInputs,
5005 uint8_t*& dstp, int dst_stride,
5006 std::vector<const uint8_t*>& srcp, std::vector<int>& src_stride, std::vector<intptr_t>& ptroffsets, std::vector<const uint8_t*>& srcp_orig)
5007 {
5008 #ifdef VS_TARGET_CPU_X86
5009
3/4
✓ Branch 2 → 3 taken 13 times.
✓ Branch 2 → 30 taken 20 times.
✓ Branch 3 → 4 taken 13 times.
✗ Branch 3 → 30 not taken.
 33 if (optSSE2 && d.planeOptSSE2[plane]) {
5010
5011 13 int nfulliterations = w / pixels_per_iter;
5012
5013 13 ExprData::ProcessLineProc proc = d.proc[plane];
5014
5015 alignas(32) intptr_t rwptrs[RWPTR_SIZE]; // should work, gcc 8.3 gives false warning
5016
5017 13 *reinterpret_cast<float*>(&rwptrs[RWPTR_START_OF_INTERNAL_VARIABLES + INTERNAL_VAR_CURRENT_FRAME]) = (float)framecount;
5018 13 *reinterpret_cast<float*>(&rwptrs[RWPTR_START_OF_INTERNAL_VARIABLES + INTERNAL_VAR_RELTIME]) = (float)relative_time;
5019 // refresh frame properties
5020
1/2
✗ Branch 17 → 6 not taken.
✓ Branch 17 → 18 taken 13 times.
 26 for (auto& framePropToRead : d.frameprops[plane]) {
5021 ✗ int whereToPut = framePropToRead.var_index;
5022 ✗ *reinterpret_cast<float*>(&rwptrs[RWPTR_START_OF_INTERNAL_FRAMEPROP_VARIABLES + whereToPut]) = framePropToRead.value;
5023 };
5024
2/2
✓ Branch 28 → 19 taken 49 times.
✓ Branch 28 → 29 taken 13 times.
 62 for (int y = 0; y < h; y++) {
5025 49 rwptrs[RWPTR_START_OF_OUTPUT] = reinterpret_cast<intptr_t>(dstp + dst_stride * y);
5026 49 rwptrs[RWPTR_START_OF_XCOUNTER] = 0; // xcounter internal variable
5027
2/2
✓ Branch 24 → 20 taken 49 times.
✓ Branch 24 → 25 taken 49 times.
 98 for (int i = 0; i < numInputs; i++) {
5028 49 rwptrs[i + RWPTR_START_OF_INPUTS] = reinterpret_cast<intptr_t>(srcp[i] + src_stride[i] * y); // input pointers 1..Nth
5029 49 rwptrs[i + RWPTR_START_OF_STRIDES] = static_cast<intptr_t>(src_stride[i]);
5030 }
5031 // a single line at a time
5032
1/2
✓ Branch 26 → 27 taken 49 times.
✗ Branch 26 → 290 not taken.
 49 proc(rwptrs, ptroffsets.data(), nfulliterations, y); // parameters are put directly in registers
5033 }
5034 13 }
5035 else
5036 #endif // VS_TARGET_CPU_X86
5037
2/2
✓ Branch 30 → 31 taken 8 times.
✓ Branch 30 → 32 taken 12 times.
 20 if (optVectorC) {
5038 // SIMD factory, vector friendly C version 16 then 8, 4, 1 floats at a time
5039 // Even if the compiler does not vectorize, we have less overhead during the opcode flow processing
5040 // As of 2025: original:1-2.5fps, new MAX_C_VECT=16: MSVC~6fps MSVC AVX2:~6fps, LLVM-14,7fps, LLVM AVX2-19fps
5041
5042 8 processFrameWithDynamicVectors<MAX_C_VECT>(
5043 plane, w, h, pixels_per_iter, framecount, relative_time, numInputs,
5044 dstp, dst_stride,
5045 8 srcp, src_stride, ptroffsets, srcp_orig, d);
5046 }
5047 else
5048 {
5049 // C version, single pixel/loop reference
5050
1/2
✓ Branch 34 → 35 taken 12 times.
✗ Branch 34 → 291 not taken.
 12 std::vector<float> stackVector(d.maxStackSize);
5051
5052 12 const ExprOp* vops = d.ops[plane].data();
5053 12 float* stack = stackVector.data();
5054 12 float stacktop = 0;
5055
5056
1/2
✓ Branch 40 → 41 taken 12 times.
✗ Branch 40 → 294 not taken.
 24 std::vector<float> variable_area(MAX_USER_VARIABLES); // for C, place for expr variables A..Z
5057
1/2
✓ Branch 44 → 45 taken 12 times.
✗ Branch 44 → 297 not taken.
 12 std::vector<float> internal_vars(INTERNAL_VARIABLES + MAX_FRAMEPROP_VARIABLES);
5058 12 internal_vars[INTERNAL_VAR_CURRENT_FRAME] = (float)framecount;
5059 12 internal_vars[INTERNAL_VAR_RELTIME] = (float)relative_time;
5060 // followed by dynamic frame properties
5061
1/2
✗ Branch 62 → 50 not taken.
✓ Branch 62 → 63 taken 12 times.
 24 for (auto& framePropToRead : d.frameprops[plane]) {
5062 ✗ int whereToPut = framePropToRead.var_index;
5063 ✗ internal_vars[INTERNAL_VAR_FRAMEPROP_VARIABLES_START + whereToPut] = framePropToRead.value;
5064 };
5065
5066
2/2
✓ Branch 284 → 64 taken 37 times.
✓ Branch 284 → 285 taken 12 times.
 49 for (int y = 0; y < h; y++) {
5067
2/2
✓ Branch 276 → 65 taken 614 times.
✓ Branch 276 → 277 taken 37 times.
 651 for (int x = 0; x < w; x++) {
5068 614 int si = 0;
5069 614 int i = -1;
5070 while (true) {
5071 2526 i++;
5072
12/60
✓ Branch 66 → 67 taken 230 times.
✓ Branch 66 → 68 taken 230 times.
✗ Branch 66 → 69 not taken.
✗ Branch 66 → 71 not taken.
✓ Branch 66 → 73 taken 377 times.
✓ Branch 66 → 75 taken 153 times.
✓ Branch 66 → 77 taken 8 times.
✓ Branch 66 → 79 taken 76 times.
✗ Branch 66 → 102 not taken.
✗ Branch 66 → 125 not taken.
✓ Branch 66 → 148 taken 185 times.
✗ Branch 66 → 149 not taken.
✗ Branch 66 → 151 not taken.
✗ Branch 66 → 152 not taken.
✓ Branch 66 → 154 taken 645 times.
✗ Branch 66 → 155 not taken.
✗ Branch 66 → 156 not taken.
✗ Branch 66 → 157 not taken.
✗ Branch 66 → 158 not taken.
✗ Branch 66 → 160 not taken.
✗ Branch 66 → 162 not taken.
✗ Branch 66 → 164 not taken.
✗ Branch 66 → 166 not taken.
✗ Branch 66 → 168 not taken.
✗ Branch 66 → 170 not taken.
✗ Branch 66 → 173 not taken.
✗ Branch 66 → 175 not taken.
✗ Branch 66 → 177 not taken.
✗ Branch 66 → 179 not taken.
✓ Branch 66 → 181 taken 8 times.
✗ Branch 66 → 183 not taken.
✗ Branch 66 → 185 not taken.
✗ Branch 66 → 191 not taken.
✗ Branch 66 → 193 not taken.
✗ Branch 66 → 195 not taken.
✗ Branch 66 → 197 not taken.
✗ Branch 66 → 199 not taken.
✗ Branch 66 → 201 not taken.
✗ Branch 66 → 203 not taken.
✗ Branch 66 → 205 not taken.
✗ Branch 66 → 209 not taken.
✗ Branch 66 → 213 not taken.
✗ Branch 66 → 217 not taken.
✗ Branch 66 → 221 not taken.
✗ Branch 66 → 225 not taken.
✗ Branch 66 → 229 not taken.
✗ Branch 66 → 233 not taken.
✗ Branch 66 → 238 not taken.
✗ Branch 66 → 243 not taken.
✗ Branch 66 → 247 not taken.
✗ Branch 66 → 251 not taken.
✓ Branch 66 → 252 taken 453 times.
✗ Branch 66 → 255 not taken.
✗ Branch 66 → 258 not taken.
✗ Branch 66 → 261 not taken.
✓ Branch 66 → 264 taken 153 times.
✓ Branch 66 → 267 taken 8 times.
✗ Branch 66 → 268 not taken.
✗ Branch 66 → 270 not taken.
✗ Branch 66 → 274 not taken.
 2526 switch (vops[i].op) {
5073 230 case opLoadSpatialX:
5074 230 stack[si] = stacktop;
5075 230 stacktop = (float)x;
5076 230 ++si;
5077 230 break;
5078 230 case opLoadSpatialY:
5079 230 stack[si] = stacktop;
5080 230 stacktop = (float)y;
5081 230 ++si;
5082 230 break;
5083 ✗ case opLoadInternalVar:
5084 ✗ stack[si] = stacktop;
5085 ✗ stacktop = internal_vars[vops[i].e.ival];
5086 ✗ ++si;
5087 ✗ break;
5088 ✗ case opLoadFramePropVar:
5089 ✗ stack[si] = stacktop;
5090 ✗ stacktop = internal_vars[INTERNAL_VAR_FRAMEPROP_VARIABLES_START + vops[i].e.ival];
5091 ✗ ++si;
5092 ✗ break;
5093 377 case opLoadSrc8:
5094 377 stack[si] = stacktop;
5095 377 stacktop = srcp[vops[i].e.ival][x];
5096 377 ++si;
5097 377 break;
5098 153 case opLoadSrc16:
5099 153 stack[si] = stacktop;
5100 153 stacktop = reinterpret_cast<const uint16_t*>(srcp[vops[i].e.ival])[x];
5101 153 ++si;
5102 153 break;
5103 8 case opLoadSrcF32:
5104 8 stack[si] = stacktop;
5105 8 stacktop = reinterpret_cast<const float*>(srcp[vops[i].e.ival])[x];
5106 8 ++si;
5107 8 break;
5108 76 case opLoadRelSrc8:
5109 76 stack[si] = stacktop;
5110 {
5111 76 const int newx = x + vops[i].dx;
5112 76 const int newy = y + vops[i].dy;
5113 76 const int clipIndex = vops[i].e.ival;
5114
3/4
✓ Branch 80 → 81 taken 76 times.
✗ Branch 80 → 82 not taken.
✓ Branch 85 → 86 taken 19 times.
✓ Branch 85 → 87 taken 57 times.
 228 const uint8_t* srcp2 = srcp_orig[clipIndex] + max(0, min(newy, h - 1)) * src_stride[clipIndex];
5115
3/4
✓ Branch 91 → 92 taken 72 times.
✓ Branch 91 → 93 taken 4 times.
✗ Branch 96 → 97 not taken.
✓ Branch 96 → 98 taken 76 times.
 152 stacktop = srcp2[max(0, min(newx, w - 1))];
5116 }
5117 76 ++si;
5118 76 break;
5119 ✗ case opLoadRelSrc16:
5120 ✗ stack[si] = stacktop;
5121 {
5122 ✗ const int newx = x + vops[i].dx;
5123 ✗ const int newy = y + vops[i].dy;
5124 ✗ const int clipIndex = vops[i].e.ival;
5125 ✗ const uint16_t* srcp2 = reinterpret_cast<const uint16_t*>(srcp_orig[clipIndex] + max(0, min(newy, h - 1)) * src_stride[clipIndex]);
5126 ✗ stacktop = srcp2[max(0, min(newx, w - 1))];
5127 }
5128 ✗ ++si;
5129 ✗ break;
5130 ✗ case opLoadRelSrcF32:
5131 ✗ stack[si] = stacktop;
5132 {
5133 ✗ const int newx = x + vops[i].dx;
5134 ✗ const int newy = y + vops[i].dy;
5135 ✗ const int clipIndex = vops[i].e.ival;
5136 ✗ const float* srcp2 = reinterpret_cast<const float*>(srcp_orig[clipIndex] + max(0, min(newy, h - 1)) * src_stride[clipIndex]);
5137 ✗ stacktop = srcp2[max(0, min(newx, w - 1))];
5138 }
5139 ✗ ++si;
5140 ✗ break;
5141 185 case opLoadConst:
5142 185 stack[si] = stacktop;
5143 185 stacktop = vops[i].e.fval;
5144 185 ++si;
5145 185 break;
5146 ✗ case opLoadVar:
5147 ✗ stack[si] = stacktop;
5148 ✗ stacktop = variable_area[vops[i].e.ival];
5149 ✗ ++si;
5150 ✗ break;
5151 ✗ case opDup:
5152 ✗ stack[si] = stacktop;
5153 ✗ stacktop = stack[si - vops[i].e.ival];
5154 ✗ ++si;
5155 ✗ break;
5156 ✗ case opSwap:
5157 ✗ std::swap(stacktop, stack[si - vops[i].e.ival]);
5158 ✗ break;
5159 645 case opAdd:
5160 645 --si;
5161 645 stacktop += stack[si];
5162 645 break;
5163 ✗ case opSub:
5164 ✗ --si;
5165 ✗ stacktop = stack[si] - stacktop;
5166 ✗ break;
5167 ✗ case opMul:
5168 ✗ --si;
5169 ✗ stacktop *= stack[si];
5170 ✗ break;
5171 ✗ case opDiv:
5172 ✗ --si;
5173 ✗ stacktop = stack[si] / stacktop;
5174 ✗ break;
5175 ✗ case opFmod:
5176 ✗ --si;
5177 ✗ stacktop = std::fmod(stack[si], stacktop);
5178 ✗ break;
5179 ✗ case opMax:
5180 ✗ --si;
5181 ✗ stacktop = std::max(stacktop, stack[si]);
5182 ✗ break;
5183 ✗ case opMin:
5184 ✗ --si;
5185 ✗ stacktop = std::min(stacktop, stack[si]);
5186 ✗ break;
5187 ✗ case opExp:
5188 ✗ stacktop = std::exp(stacktop);
5189 ✗ break;
5190 ✗ case opLog:
5191 ✗ stacktop = std::log(stacktop);
5192 ✗ break;
5193 ✗ case opPow:
5194 ✗ --si;
5195 ✗ stacktop = std::pow(stack[si], stacktop);
5196 ✗ break;
5197 ✗ case opClip:
5198 // clip(a, low, high) = min(max(a, low),high)
5199 ✗ si -= 2;
5200 ✗ stacktop = std::max(std::min(stack[si], stacktop), stack[si + 1]);
5201 ✗ break;
5202 ✗ case opRound:
5203 ✗ stacktop = std::round(stacktop);
5204 ✗ break;
5205 ✗ case opFloor:
5206 ✗ stacktop = std::floor(stacktop);
5207 ✗ break;
5208 ✗ case opCeil:
5209 ✗ stacktop = std::ceil(stacktop);
5210 ✗ break;
5211 ✗ case opTrunc:
5212 ✗ stacktop = std::trunc(stacktop);
5213 ✗ break;
5214 8 case opSqrt:
5215 8 stacktop = std::sqrt(stacktop);
5216 8 break;
5217 ✗ case opAbs:
5218 ✗ stacktop = std::abs(stacktop);
5219 ✗ break;
5220 ✗ case opSgn:
5221 ✗ stacktop = stacktop < 0 ? -1.0f : stacktop > 0 ? 1.0f : 0.0f;
5222 ✗ break;
5223 ✗ case opSin:
5224 ✗ stacktop = std::sin(stacktop);
5225 ✗ break;
5226 ✗ case opCos:
5227 ✗ stacktop = std::cos(stacktop);
5228 ✗ break;
5229 ✗ case opTan:
5230 ✗ stacktop = std::tan(stacktop);
5231 ✗ break;
5232 ✗ case opAsin:
5233 ✗ stacktop = std::asin(stacktop);
5234 ✗ break;
5235 ✗ case opAcos:
5236 ✗ stacktop = std::acos(stacktop);
5237 ✗ break;
5238 ✗ case opAtan:
5239 ✗ stacktop = std::atan(stacktop);
5240 ✗ break;
5241 ✗ case opAtan2:
5242 ✗ --si;
5243 ✗ stacktop = std::atan2(stack[si], stacktop); // y, x -> -Pi..+Pi
5244 ✗ break;
5245 ✗ case opGt:
5246 ✗ --si;
5247 ✗ stacktop = (stack[si] > stacktop) ? 1.0f : 0.0f;
5248 ✗ break;
5249 ✗ case opLt:
5250 ✗ --si;
5251 ✗ stacktop = (stack[si] < stacktop) ? 1.0f : 0.0f;
5252 ✗ break;
5253 ✗ case opEq:
5254 ✗ --si;
5255 ✗ stacktop = (stack[si] == stacktop) ? 1.0f : 0.0f;
5256 ✗ break;
5257 ✗ case opNotEq:
5258 ✗ --si;
5259 ✗ stacktop = (stack[si] != stacktop) ? 1.0f : 0.0f;
5260 ✗ break;
5261 ✗ case opLE:
5262 ✗ --si;
5263 ✗ stacktop = (stack[si] <= stacktop) ? 1.0f : 0.0f;
5264 ✗ break;
5265 ✗ case opGE:
5266 ✗ --si;
5267 ✗ stacktop = (stack[si] >= stacktop) ? 1.0f : 0.0f;
5268 ✗ break;
5269 ✗ case opTernary:
5270 ✗ si -= 2;
5271 ✗ stacktop = (stack[si] > 0) ? stack[si + 1] : stacktop;
5272 ✗ break;
5273 ✗ case opAnd:
5274 ✗ --si;
5275 ✗ stacktop = (stacktop > 0 && stack[si] > 0) ? 1.0f : 0.0f;
5276 ✗ break;
5277 ✗ case opOr:
5278 ✗ --si;
5279 ✗ stacktop = (stacktop > 0 || stack[si] > 0) ? 1.0f : 0.0f;
5280 ✗ break;
5281 ✗ case opXor:
5282 ✗ --si;
5283 ✗ stacktop = ((stacktop > 0) != (stack[si] > 0)) ? 1.0f : 0.0f;
5284 ✗ break;
5285 ✗ case opNeg:
5286 ✗ stacktop = (stacktop > 0) ? 0.0f : 1.0f;
5287 ✗ break;
5288 ✗ case opNegSign:
5289 ✗ stacktop = -stacktop;
5290 ✗ break;
5291 453 case opStore8:
5292 453 dstp[x] = (uint8_t)(std::max(0.0f, std::min(stacktop, 255.0f)) + 0.5f);
5293 453 goto loopend;
5294 ✗ case opStore10:
5295 ✗ reinterpret_cast<uint16_t*>(dstp)[x] = (uint16_t)(std::max(0.0f, std::min(stacktop, 1023.0f)) + 0.5f);
5296 ✗ goto loopend;
5297 ✗ case opStore12:
5298 ✗ reinterpret_cast<uint16_t*>(dstp)[x] = (uint16_t)(std::max(0.0f, std::min(stacktop, 4095.0f)) + 0.5f);
5299 ✗ goto loopend;
5300 ✗ case opStore14:
5301 ✗ reinterpret_cast<uint16_t*>(dstp)[x] = (uint16_t)(std::max(0.0f, std::min(stacktop, 16383.0f)) + 0.5f);
5302 ✗ goto loopend;
5303 153 case opStore16:
5304 153 reinterpret_cast<uint16_t*>(dstp)[x] = (uint16_t)(std::max(0.0f, std::min(stacktop, 65535.0f)) + 0.5f);
5305 153 goto loopend;
5306 8 case opStoreF32:
5307 8 reinterpret_cast<float*>(dstp)[x] = stacktop;
5308 8 goto loopend;
5309 ✗ case opStoreVar:
5310 ✗ variable_area[vops[i].e.ival] = stacktop;
5311 ✗ break;
5312 ✗ case opStoreVarAndDrop1:
5313 ✗ variable_area[vops[i].e.ival] = stacktop;
5314 ✗ --si;
5315 ✗ if (si >= 0)
5316 ✗ stacktop = stack[si];
5317 ✗ break;
5318 }
5319 1912 }
5320 614 loopend:;
5321 }
5322 37 dstp += dst_stride;
5323
1/2
✓ Branch 277 → 278 taken 37 times.
✗ Branch 277 → 283 not taken.
 37 if (d.lutmode == 0) {
5324
2/2
✓ Branch 282 → 279 taken 37 times.
✓ Branch 282 → 283 taken 37 times.
 74 for (int i = 0; i < numInputs; i++)
5325 37 srcp[i] += src_stride[i];
5326 }
5327 }
5328 12 }
5329 33 }
5330
5331 34 void Exprfilter::preReadFrameProps(int plane, std::vector<PVideoFrame>& src, IScriptEnvironment* env)
5332 {
5333
1/2
✗ Branch 31 → 4 not taken.
✓ Branch 31 → 32 taken 34 times.
 68 for (auto& framePropToRead : d.frameprops[plane]) {
5334 ✗ int srcIndex = framePropToRead.srcIndex;
5335 ✗ auto fpname = framePropToRead.name;
5336
5337 ✗ const AVSMap* avsmap = env->getFramePropsRO(src[srcIndex]);
5338
5339 // default is 0f
5340 ✗ float varToStore = 0.0f; // std::numeric_limits<float>::quiet_NaN();
5341
5342 ✗ char res = env->propGetType(avsmap, fpname.c_str());
5343 // 'u'nset, 'i'nteger, 'f'loat, 's'string, 'c'lip, 'v'ideoframe, 'm'ethod };
5344
5345 int error;
5346 // only float and int are valid
5347 // cast to float: Expr supports 32 bit float, no double,
5348 ✗ if (res == 'i') {
5349 ✗ int64_t result = env->propGetInt(avsmap, fpname.c_str(), 0, &error);
5350 ✗ if (!error) varToStore = static_cast<float>(result);
5351 }
5352 ✗ else if (res == 'f') {
5353 ✗ float result = env->propGetFloatSaturated(avsmap, fpname.c_str(), 0, &error);
5354 ✗ if (!error) varToStore = result;
5355 }
5356
5357 ✗ framePropToRead.value = varToStore;
5358 ✗ }
5359 34 }
5360
5361 1 void Exprfilter::calculate_lut(IScriptEnvironment* env)
5362 {
5363 // ExprData d class variable already filled
5364
5365 // Only when there are frame props.
5366 // frame property set from GetFrame(0) is treated as Clip prop
5367
5368 1 std::vector<PVideoFrame> src;
5369
5370 1 bool frameprops = false;
5371
3/4
✓ Branch 8 → 9 taken 2 times.
✗ Branch 8 → 105 not taken.
✓ Branch 9 → 4 taken 1 time.
✓ Branch 9 → 10 taken 1 time.
 2 for (int plane = 0; plane < d.vi.NumComponents(); plane++) {
5372
1/2
✗ Branch 5 → 6 not taken.
✓ Branch 5 → 7 taken 1 time.
 1 if (d.frameprops[plane].size() > 0) {
5373 ✗ frameprops = true;
5374 ✗ break;
5375 }
5376 }
5377
5378
1/2
✗ Branch 10 → 11 not taken.
✓ Branch 10 → 22 taken 1 time.
 1 if (frameprops) {
5379 ✗ src.reserve(children.size());
5380 // fetch 0th frame only when needed in lut
5381 ✗ for (size_t i = 0; i < children.size(); i++) {
5382 ✗ const auto& child = children[i];
5383 ✗ src.emplace_back(child->GetFrame(0, env));
5384 }
5385 }
5386
5387
1/2
✓ Branch 24 → 25 taken 1 time.
✗ Branch 24 → 84 not taken.
 2 std::vector<const uint8_t*> srcp(MAX_EXPR_INPUTS);
5388
1/2
✓ Branch 28 → 29 taken 1 time.
✗ Branch 28 → 87 not taken.
 2 std::vector<const uint8_t*> srcp_orig(MAX_EXPR_INPUTS);
5389
1/2
✓ Branch 32 → 33 taken 1 time.
✗ Branch 32 → 90 not taken.
 1 std::vector<int> src_stride(MAX_EXPR_INPUTS);
5390
5391
3/4
✓ Branch 74 → 75 taken 2 times.
✗ Branch 74 → 99 not taken.
✓ Branch 75 → 35 taken 1 time.
✓ Branch 75 → 76 taken 1 time.
 2 for (int plane = 0; plane < d.vi.NumComponents(); plane++) {
5392 // calculate only if plane is processed
5393
1/2
✗ Branch 35 → 36 not taken.
✓ Branch 35 → 37 taken 1 time.
 1 if (d.plane[plane] != poProcess)
5394 ✗ continue;
5395
5396 // read actually needed frame properties into the variable storage area
5397
1/2
✓ Branch 37 → 38 taken 1 time.
✗ Branch 37 → 98 not taken.
 1 preReadFrameProps(plane, src, env);
5398
5399 uint8_t* dstp;
5400 int dst_stride;
5401 int h, w;
5402
5403 // no buffer allocated yet, prepare lut target buffer, and fake input frame dimensions
5404
1/2
✓ Branch 38 → 39 taken 1 time.
✗ Branch 38 → 98 not taken.
 1 const int bits_per_pixel = d.vi.BitsPerComponent();
5405
1/2
✓ Branch 39 → 40 taken 1 time.
✗ Branch 39 → 98 not taken.
 1 const int pixelsize = d.vi.ComponentSize();
5406 1 const auto lut1d_size = (1 << bits_per_pixel); // 1 or 2 bytes per entry
5407 1 const auto lut1d_bytesize = lut1d_size * pixelsize;
5408
1/2
✗ Branch 40 → 41 not taken.
✓ Branch 40 → 42 taken 1 time.
 1 const auto lut_size = d.lutmode == 1 ? lut1d_bytesize : lut1d_bytesize * lut1d_bytesize;
5409 // buffer start must be aligned to at least 32 bytes for avx2.
5410 // Size must be mod64 but it is fulfilled always.
5411 1 d.luts[plane] = (uint8_t *)avs_malloc(lut_size, 32); // 256 lut_x 65536: lut_xy (8 bit)
5412 1 dstp = d.luts[plane];
5413 1 dst_stride = lut1d_bytesize;
5414
1/2
✗ Branch 44 → 45 not taken.
✓ Branch 44 → 46 taken 1 time.
 1 h = lutmode == 1 ? 1 : lut1d_size; // 1x256, 256x256. 10 bit: 1024, 1024x1024
5415 1 w = lut1d_size;
5416
5417 // for simd:
5418 // same as in GetFrame
5419
2/8
✗ Branch 47 → 48 not taken.
✓ Branch 47 → 53 taken 1 time.
✗ Branch 48 → 49 not taken.
✗ Branch 48 → 53 not taken.
✗ Branch 49 → 50 not taken.
✗ Branch 49 → 51 not taken.
✗ Branch 53 → 54 not taken.
✓ Branch 53 → 55 taken 1 time.
 1 const int pixels_per_iter = (optAvx2 && d.planeOptAvx2[plane]) ? (optSingleMode ? 8 : 16) : (optSingleMode ? 4 : 8);
5420
1/2
✓ Branch 58 → 59 taken 1 time.
✗ Branch 58 → 93 not taken.
 1 std::vector<intptr_t> ptroffsets(1 + 1 + MAX_EXPR_INPUTS);
5421
1/2
✓ Branch 60 → 61 taken 1 time.
✗ Branch 60 → 96 not taken.
 1 ptroffsets[RWPTR_START_OF_OUTPUT] = d.vi.ComponentSize() * pixels_per_iter; // stepping for output pointer
5422 1 ptroffsets[RWPTR_START_OF_XCOUNTER] = pixels_per_iter; // stepping for xcounter
5423
5424 // no srcp pointers in lut. Technically only inputless sx,sy relative coordinates are in there
5425
2/2
✓ Branch 69 → 64 taken 1 time.
✓ Branch 69 → 70 taken 1 time.
 2 for (int i = 0; i < d.numInputs; i++) {
5426 1 srcp[i] = nullptr;
5427 1 srcp_orig[i] = nullptr;
5428 1 src_stride[i] = 0;
5429 1 ptroffsets[RWPTR_START_OF_INPUTS + i] = 0;
5430 }
5431
5432 1 const int dummy_framecount = 0;
5433 1 const int dummy_relative_time = 0;
5434
1/2
✓ Branch 70 → 71 taken 1 time.
✗ Branch 70 → 96 not taken.
 1 processFrame(plane, w, h, pixels_per_iter, dummy_framecount, dummy_relative_time, d.numInputs, dstp, dst_stride, srcp, src_stride, ptroffsets, srcp_orig);
5435 1 } // for planes
5436 1 }
5437
5438 template<typename pixel_t, int bits_per_pixel>
5439 ✗ static void do_lut_xy(const uint8_t* lut8, uint8_t* dstp, int dst_stride, const uint8_t** srcp, const int* src_stride, int w, int h)
5440 {
5441 ✗ const int max_pixel_value = (1 << bits_per_pixel) - 1;
5442 ✗ const pixel_t* lut = reinterpret_cast<const pixel_t*>(lut8);
5443 ✗ const uint8_t* src0 = srcp[0];
5444 ✗ const uint8_t* src1 = srcp[1];
5445 ✗ const auto pitch0 = src_stride[0];
5446 ✗ const auto pitch1 = src_stride[1];
5447 ✗ for (auto y = 0; y < h; y++) {
5448 ✗ for (auto x = 0; x < w; x++) {
5449 if constexpr (bits_per_pixel == 8 || bits_per_pixel == 16) {
5450 // no limit check
5451 ✗ const int pixel0 = reinterpret_cast<const pixel_t*>(src0)[x];
5452 ✗ const int pixel1 = reinterpret_cast<const pixel_t*>(src1)[x];
5453 ✗ reinterpret_cast<pixel_t*>(dstp)[x] = lut[(pixel1 << bits_per_pixel) + pixel0];
5454 }
5455 else {
5456 ✗ const int pixel0 = min((int)reinterpret_cast<const pixel_t*>(src0)[x], max_pixel_value);
5457 ✗ const int pixel1 = min((int)reinterpret_cast<const pixel_t*>(src1)[x], max_pixel_value);
5458 ✗ reinterpret_cast<pixel_t*>(dstp)[x] = lut[(pixel1 << bits_per_pixel) + pixel0];
5459 }
5460 }
5461 ✗ src0 += pitch0;
5462 ✗ src1 += pitch1;
5463 ✗ dstp += dst_stride;
5464 }
5465 ✗ }
5466
5467 15 PVideoFrame __stdcall Exprfilter::GetFrame(int n, IScriptEnvironment *env) {
5468 // ExprData d class variable already filled
5469
5470 15 std::vector<PVideoFrame> src;
5471
1/2
✓ Branch 4 → 5 taken 15 times.
✗ Branch 4 → 258 not taken.
 15 src.reserve(children.size());
5472
5473 15 int first_used_clip_index = -1;
5474
2/2
✓ Branch 18 → 6 taken 15 times.
✓ Branch 18 → 19 taken 15 times.
 30 for (size_t i = 0; i < children.size(); i++) {
5475 15 const auto &child = children[i];
5476
3/8
✓ Branch 7 → 8 taken 15 times.
✗ Branch 7 → 10 not taken.
✓ Branch 9 → 11 taken 15 times.
✗ Branch 9 → 227 not taken.
✗ Branch 10 → 11 not taken.
✗ Branch 10 → 227 not taken.
✓ Branch 11 → 12 taken 15 times.
✗ Branch 11 → 225 not taken.
 15 src.emplace_back(d.clipsUsed[i] ? child->GetFrame(n, env) : nullptr); // GetFrame only when really referenced
5477
1/2
✓ Branch 13 → 14 taken 15 times.
✗ Branch 13 → 16 not taken.
 15 if (first_used_clip_index < 0) {
5478
1/2
✓ Branch 14 → 15 taken 15 times.
✗ Branch 14 → 16 not taken.
 15 if (d.clipsUsed[i])
5479 15 first_used_clip_index = (int)i; // inherit frameprop from
5480 }
5481 }
5482
5483
1/2
✓ Branch 19 → 20 taken 15 times.
✗ Branch 19 → 258 not taken.
 15 PVideoFrame dst;
5484
1/2
✓ Branch 20 → 21 taken 15 times.
✗ Branch 20 → 26 not taken.
 15 if (first_used_clip_index >= 0)
5485
2/4
✓ Branch 22 → 23 taken 15 times.
✗ Branch 22 → 230 not taken.
✓ Branch 23 → 24 taken 15 times.
✗ Branch 23 → 228 not taken.
 15 dst = env->NewVideoFrameP(d.vi, &src[first_used_clip_index]);
5486 else
5487 ✗ dst = env->NewVideoFrame(d.vi);
5488
5489
1/2
✓ Branch 32 → 33 taken 15 times.
✗ Branch 32 → 234 not taken.
 30 std::vector<const uint8_t*> srcp(MAX_EXPR_INPUTS);
5490
1/2
✓ Branch 36 → 37 taken 15 times.
✗ Branch 36 → 237 not taken.
 30 std::vector<const uint8_t*> srcp_orig(MAX_EXPR_INPUTS);
5491
1/2
✓ Branch 40 → 41 taken 15 times.
✗ Branch 40 → 240 not taken.
 15 std::vector<int> src_stride(MAX_EXPR_INPUTS);
5492
5493 15 const float framecount = (float)n; // max precision: 2^24 (16M) frames (32 bit float precision)
5494
2/2
✓ Branch 42 → 43 taken 5 times.
✓ Branch 42 → 44 taken 10 times.
 15 const float relative_time = vi.num_frames > 1 ? (float)((double)n / (vi.num_frames - 1)) : 0.0f; // 0 <= time <= 1
5495 15 const int planes_y[4] = { PLANAR_Y, PLANAR_U, PLANAR_V, PLANAR_A };
5496 15 const int planes_r[4] = { PLANAR_R, PLANAR_G, PLANAR_B, PLANAR_A }; // expression string order is R G B unlike internal G B R plane order
5497
6/8
✓ Branch 45 → 46 taken 15 times.
✗ Branch 45 → 250 not taken.
✓ Branch 46 → 47 taken 6 times.
✓ Branch 46 → 49 taken 9 times.
✓ Branch 47 → 48 taken 6 times.
✗ Branch 47 → 250 not taken.
✓ Branch 48 → 49 taken 2 times.
✓ Branch 48 → 50 taken 4 times.
 15 const int *plane_enums_d = (d.vi.IsYUV() || d.vi.IsYUVA()) ? planes_y : planes_r;
5498
5499
3/4
✓ Branch 217 → 218 taken 56 times.
✗ Branch 217 → 250 not taken.
✓ Branch 218 → 52 taken 41 times.
✓ Branch 218 → 219 taken 15 times.
 56 for (int plane = 0; plane < d.vi.NumComponents(); plane++) {
5500
5501 41 const int plane_enum_d = plane_enums_d[plane];
5502
5503
2/2
✓ Branch 52 → 53 taken 33 times.
✓ Branch 52 → 172 taken 8 times.
 41 if (d.plane[plane] == poProcess) {
5504
5505 // read actually needed frame properties into the variable storage area
5506
1/2
✓ Branch 53 → 54 taken 33 times.
✗ Branch 53 → 248 not taken.
 33 preReadFrameProps(plane, src, env);
5507
5508 uint8_t* dstp;
5509 int dst_stride;
5510 int h, w;
5511
5512
1/2
✓ Branch 55 → 56 taken 33 times.
✗ Branch 55 → 248 not taken.
 33 dstp = dst->GetWritePtr(plane_enum_d);
5513
1/2
✓ Branch 57 → 58 taken 33 times.
✗ Branch 57 → 248 not taken.
 33 dst_stride = dst->GetPitch(plane_enum_d);
5514
1/2
✓ Branch 58 → 59 taken 33 times.
✗ Branch 58 → 248 not taken.
 33 h = d.vi.height >> d.vi.GetPlaneHeightSubsampling(plane_enum_d);
5515
1/2
✓ Branch 59 → 60 taken 33 times.
✗ Branch 59 → 248 not taken.
 33 w = d.vi.width >> d.vi.GetPlaneWidthSubsampling(plane_enum_d);
5516
5517 // for simd:
5518
5/8
✓ Branch 60 → 61 taken 6 times.
✓ Branch 60 → 66 taken 27 times.
✓ Branch 61 → 62 taken 6 times.
✗ Branch 61 → 66 not taken.
✗ Branch 62 → 63 not taken.
✓ Branch 62 → 64 taken 6 times.
✗ Branch 66 → 67 not taken.
✓ Branch 66 → 68 taken 27 times.
 33 const int pixels_per_iter = (optAvx2 && d.planeOptAvx2[plane]) ? (optSingleMode ? 8 : 16) : (optSingleMode ? 4 : 8);
5519
1/2
✓ Branch 71 → 72 taken 33 times.
✗ Branch 71 → 243 not taken.
 33 std::vector<intptr_t> ptroffsets(1 + 1 + MAX_EXPR_INPUTS);
5520
1/2
✓ Branch 73 → 74 taken 33 times.
✗ Branch 73 → 246 not taken.
 33 ptroffsets[RWPTR_START_OF_OUTPUT] = d.vi.ComponentSize() * pixels_per_iter; // stepping for output pointer
5521 33 ptroffsets[RWPTR_START_OF_XCOUNTER] = pixels_per_iter; // stepping for xcounter
5522
5523
2/2
✓ Branch 113 → 77 taken 33 times.
✓ Branch 113 → 114 taken 33 times.
 66 for (int i = 0; i < d.numInputs; i++) {
5524
1/2
✓ Branch 78 → 79 taken 33 times.
✗ Branch 78 → 112 not taken.
 33 if (d.clips[i]) {
5525
1/2
✓ Branch 79 → 80 taken 33 times.
✗ Branch 79 → 107 not taken.
 33 if (d.clipsUsed[i]) {
5526 // when input is a single Y, use PLANAR_Y instead of the plane matching to the output
5527
1/2
✓ Branch 81 → 82 taken 33 times.
✗ Branch 81 → 246 not taken.
 33 const VideoInfo& vi_src = d.clips[i]->GetVideoInfo();
5528
5/8
✓ Branch 82 → 83 taken 33 times.
✗ Branch 82 → 246 not taken.
✓ Branch 83 → 84 taken 16 times.
✓ Branch 83 → 86 taken 17 times.
✓ Branch 84 → 85 taken 16 times.
✗ Branch 84 → 246 not taken.
✗ Branch 85 → 86 not taken.
✓ Branch 85 → 87 taken 16 times.
 33 const int* plane_enums_s = (vi_src.IsYUV() || vi_src.IsYUVA()) ? planes_y : planes_r;
5529
3/4
✓ Branch 88 → 89 taken 33 times.
✗ Branch 88 → 246 not taken.
✓ Branch 89 → 90 taken 5 times.
✓ Branch 89 → 91 taken 28 times.
 33 const int plane_enum_s = vi_src.IsY() ? PLANAR_Y : plane_enums_s[plane];
5530
5531
1/2
✓ Branch 94 → 95 taken 33 times.
✗ Branch 94 → 246 not taken.
 33 srcp[i] = src[i]->GetReadPtr(plane_enum_s);
5532 // C only:
5533 33 srcp_orig[i] = srcp[i];
5534
1/2
✓ Branch 100 → 101 taken 33 times.
✗ Branch 100 → 246 not taken.
 33 src_stride[i] = src[i]->GetPitch(plane_enum_s);
5535 // SIMD only
5536
2/4
✓ Branch 103 → 104 taken 33 times.
✗ Branch 103 → 246 not taken.
✓ Branch 104 → 105 taken 33 times.
✗ Branch 104 → 246 not taken.
 33 ptroffsets[RWPTR_START_OF_INPUTS + i] = d.clips[i]->GetVideoInfo().ComponentSize() * pixels_per_iter; // 1..Nth: inputs
5537 }
5538 else {
5539 ✗ srcp[i] = nullptr;
5540 ✗ srcp_orig[i] = nullptr;
5541 ✗ src_stride[i] = 0;
5542 ✗ ptroffsets[RWPTR_START_OF_INPUTS + i] = 0;
5543 }
5544 }
5545 }
5546
5547
2/2
✓ Branch 114 → 115 taken 32 times.
✓ Branch 114 → 116 taken 1 time.
 33 if (lutmode == 0) {
5548
1/2
✓ Branch 115 → 170 taken 32 times.
✗ Branch 115 → 246 not taken.
 32 processFrame(plane, w, h, pixels_per_iter, framecount, relative_time, d.numInputs, dstp, dst_stride, srcp, src_stride, ptroffsets, srcp_orig);
5549 } else {
5550 // lut table for plane is filled, do lookup now
5551
1/2
✓ Branch 116 → 117 taken 1 time.
✗ Branch 116 → 246 not taken.
 1 const int bits_per_pixel = d.vi.BitsPerComponent();
5552
5553
1/2
✓ Branch 117 → 118 taken 1 time.
✗ Branch 117 → 147 not taken.
 1 if (d.lutmode == 1) {
5554 // lut_x
5555
1/2
✓ Branch 118 → 119 taken 1 time.
✗ Branch 118 → 127 not taken.
 1 if (bits_per_pixel == 8)
5556 {
5557 1 uint8_t* lut = d.luts[plane];
5558 1 const uint8_t* src0 = srcp[0];
5559 1 const auto pitch0 = src_stride[0];
5560
2/2
✓ Branch 126 → 122 taken 2 times.
✓ Branch 126 → 170 taken 1 time.
 3 for (auto y = 0; y < h; y++) {
5561
2/2
✓ Branch 124 → 123 taken 38 times.
✓ Branch 124 → 125 taken 2 times.
 40 for (auto x = 0; x < w; x++) {
5562 38 const int pixel = src0[x];
5563 38 dstp[x] = lut[pixel];
5564 }
5565 2 src0 += pitch0;
5566 2 dstp += dst_stride;
5567 }
5568 }
5569 else {
5570 ✗ const int max_pixel_value = (1 << bits_per_pixel) - 1;
5571 ✗ uint16_t* lut = reinterpret_cast<uint16_t*>(d.luts[plane]);
5572 ✗ const uint8_t* src0 = srcp[0];
5573 ✗ const auto pitch0 = src_stride[0];
5574 ✗ if (bits_per_pixel == 16) {
5575 // no limit check
5576 ✗ for (auto y = 0; y < h; y++) {
5577 ✗ for (auto x = 0; x < w; x++) {
5578 ✗ const int pixel = reinterpret_cast<const uint16_t*>(src0)[x];
5579 ✗ reinterpret_cast<uint16_t*>(dstp)[x] = lut[pixel];
5580 }
5581 ✗ src0 += pitch0;
5582 ✗ dstp += dst_stride;
5583 }
5584 }
5585 else {
5586 ✗ for (auto y = 0; y < h; y++) {
5587 ✗ for (auto x = 0; x < w; x++) {
5588 ✗ const int pixel = reinterpret_cast<const uint16_t*>(src0)[x];
5589 ✗ reinterpret_cast<uint16_t*>(dstp)[x] = lut[min(pixel, max_pixel_value)]; // e.g. 10 bits in 2 byte safety
5590 }
5591 ✗ src0 += pitch0;
5592 ✗ dstp += dst_stride;
5593 }
5594 }
5595 }
5596 }
5597 ✗ else if (d.lutmode == 2) {
5598 // lut_xy
5599 // templates for speed: bitshift with immediate constant
5600 ✗ const uint8_t* lut = d.luts[plane];
5601 ✗ if (bits_per_pixel == 8)
5602 ✗ do_lut_xy<uint8_t, 8>(lut, dstp, dst_stride, srcp_orig.data(), src_stride.data(), w, h);
5603 ✗ else if (bits_per_pixel == 10)
5604 ✗ do_lut_xy<uint16_t, 10>(lut, dstp, dst_stride, srcp_orig.data(), src_stride.data(), w, h);
5605 ✗ else if (bits_per_pixel == 12)
5606 ✗ do_lut_xy<uint16_t, 12>(lut, dstp, dst_stride, srcp_orig.data(), src_stride.data(), w, h);
5607 ✗ else if (bits_per_pixel == 14)
5608 ✗ do_lut_xy<uint16_t, 14>(lut, dstp, dst_stride, srcp_orig.data(), src_stride.data(), w, h);
5609 ✗ else if (bits_per_pixel == 16) // well, this is not enabled 16bit lutxy would take a 8GB table
5610 ✗ do_lut_xy<uint16_t, 16>(lut, dstp, dst_stride, srcp_orig.data(), src_stride.data(), w, h);
5611 else
5612 ✗ assert(0);
5613 }
5614 else { // 1d lut, 2d lut
5615 ✗ assert(0); // no lut_xyz
5616 }
5617 } // lut branch
5618 33 }
5619 // avs+: copy plane here
5620
1/2
✓ Branch 172 → 173 taken 8 times.
✗ Branch 172 → 196 not taken.
 8 else if (d.plane[plane] == poCopy) {
5621 // avs+ copy from Nth clip
5622 8 const int copySource = d.planeCopySourceClip[plane];
5623 // when input is a single Y, use PLANAR_Y instead of the plane matching to the output
5624
1/2
✓ Branch 174 → 175 taken 8 times.
✗ Branch 174 → 250 not taken.
 8 const VideoInfo& vi_src = d.clips[copySource]->GetVideoInfo();
5625
2/4
✓ Branch 175 → 176 taken 8 times.
✗ Branch 175 → 250 not taken.
✗ Branch 176 → 177 not taken.
✓ Branch 176 → 178 taken 8 times.
 8 const int plane_enum_s = vi_src.IsY() ? PLANAR_Y : plane_enums_d[plane];
5626
5627
7/14
✓ Branch 181 → 182 taken 8 times.
✗ Branch 181 → 250 not taken.
✓ Branch 184 → 185 taken 8 times.
✗ Branch 184 → 250 not taken.
✓ Branch 187 → 188 taken 8 times.
✗ Branch 187 → 250 not taken.
✓ Branch 190 → 191 taken 8 times.
✗ Branch 190 → 250 not taken.
✓ Branch 192 → 193 taken 8 times.
✗ Branch 192 → 250 not taken.
✓ Branch 194 → 195 taken 8 times.
✗ Branch 194 → 250 not taken.
✓ Branch 195 → 216 taken 8 times.
✗ Branch 195 → 250 not taken.
 32 env->BitBlt(dst->GetWritePtr(plane_enum_d), dst->GetPitch(plane_enum_d),
5628 8 src[copySource]->GetReadPtr(plane_enum_s),
5629 8 src[copySource]->GetPitch(plane_enum_s),
5630 8 src[copySource]->GetRowSize(plane_enum_s),
5631 8 src[copySource]->GetHeight(plane_enum_s)
5632 );
5633 }
5634 ✗ else if (d.plane[plane] == poFill) { // avs+
5635 ✗ uint8_t *dstp = dst->GetWritePtr(plane_enum_d);
5636 ✗ const int dst_rowsize = dst->GetRowSize(plane_enum_d);
5637 ✗ const int dst_stride = dst->GetPitch(plane_enum_d);
5638 ✗ const int h = dst->GetHeight(plane_enum_d);
5639
5640 ✗ const int bits_per_pixel = vi.BitsPerComponent();
5641
5642 ✗ float val = d.planeFillValue[plane];
5643 ✗ int val_i = 0;
5644 ✗ if (bits_per_pixel <= 16) {
5645 ✗ const int max_pixel_value = (1 << bits_per_pixel) - 1;
5646 ✗ val_i = (int)(std::max(0.0f, std::min(val, (float)max_pixel_value)) + 0.5f);
5647 }
5648
5649 ✗ if(bits_per_pixel == 8)
5650 ✗ fill_plane<BYTE>(dstp, h, dst_rowsize, dst_stride, val_i);
5651 ✗ else if(bits_per_pixel <= 16)
5652 ✗ fill_plane<uint16_t>(dstp, h, dst_rowsize, dst_stride, val_i);
5653 else // 32 bit float
5654 ✗ fill_plane<float>(dstp, h, dst_rowsize, dst_stride, val);
5655
5656 } // plane modes
5657 } // for planes
5658
5659 15 return dst;
5660 15 }
5661
5662 30 Exprfilter::~Exprfilter() {
5663
2/2
✓ Branch 5 → 3 taken 390 times.
✓ Branch 5 → 6 taken 15 times.
 405 for (int i = 0; i < MAX_EXPR_INPUTS; i++)
5664 390 d.clips[i] = nullptr;
5665
2/2
✓ Branch 10 → 7 taken 60 times.
✓ Branch 10 → 11 taken 15 times.
 75 for (int i = 0; i < 4; i++)
5666
2/2
✓ Branch 7 → 8 taken 1 time.
✓ Branch 7 → 9 taken 59 times.
 60 if(d.luts[i]) avs_free(d.luts[i]); // aligned free
5667 15 }
5668
5669 32 static SOperation getLoadOp(const VideoInfo *vi, bool relativeKind) {
5670
1/2
✗ Branch 2 → 3 not taken.
✓ Branch 2 → 7 taken 32 times.
 32 if (!vi)
5671 ✗ return relativeKind ? opLoadRelSrcF32 : opLoadSrcF32;
5672
2/2
✓ Branch 8 → 9 taken 2 times.
✓ Branch 8 → 13 taken 30 times.
 32 if (vi->BitsPerComponent() == 32) // float, avs has no f16c float
5673
1/2
✗ Branch 9 → 10 not taken.
✓ Branch 9 → 11 taken 2 times.
 2 return relativeKind ? opLoadRelSrcF32 : opLoadSrcF32;
5674
2/2
✓ Branch 14 → 15 taken 18 times.
✓ Branch 14 → 19 taken 12 times.
 30 else if (vi->BitsPerComponent() == 8)
5675
2/2
✓ Branch 15 → 16 taken 2 times.
✓ Branch 15 → 17 taken 16 times.
 18 return relativeKind ? opLoadRelSrc8 : opLoadSrc8;
5676 else
5677
1/2
✗ Branch 19 → 20 not taken.
✓ Branch 19 → 21 taken 12 times.
 12 return relativeKind ? opLoadRelSrc16 : opLoadSrc16; // 10..16 bits common
5678 }
5679
5680 41 static SOperation getStoreOp(const VideoInfo *vi) {
5681 // avs+ has no f16c float
5682
3/7
✓ Branch 3 → 4 taken 23 times.
✗ Branch 3 → 5 not taken.
✗ Branch 3 → 6 not taken.
✗ Branch 3 → 7 not taken.
✓ Branch 3 → 8 taken 16 times.
✓ Branch 3 → 9 taken 2 times.
✗ Branch 3 → 10 not taken.
 41 switch (vi->BitsPerComponent()) {
5683 23 case 8: return opStore8;
5684 ✗ case 10: return opStore10; // avs+
5685 ✗ case 12: return opStore12; // avs+
5686 ✗ case 14: return opStore14; // avs+
5687 16 case 16: return opStore16;
5688 2 case 32: return opStoreF32;
5689 ✗ default: return opStoreF32;
5690 }
5691 }
5692
5693 #define LOAD_OP(op,v,req) do { if (stackSize < req) env->ThrowError("Expr: Not enough elements on stack to perform operation %s", tokens[i].c_str()); ops.push_back(ExprOp(op, (v))); maxStackSize = std::max(++stackSize, maxStackSize); } while(0)
5694 #define LOAD_REL_OP(op,v,req,dx,dy) do { if (stackSize < req) env->ThrowError("Expr: Not enough elements on stack to perform operation %s", tokens[i].c_str()); ops.push_back(ExprOp(op, (v), (dx), (dy))); maxStackSize = std::max(++stackSize, maxStackSize); } while(0)
5695 #define GENERAL_OP(op, v, req, dec) do { if (stackSize < req) env->ThrowError("Expr: Not enough elements on stack to perform operation %s", tokens[i].c_str()); ops.push_back(ExprOp(op, (v))); stackSize-=(dec); } while(0)
5696 #define ONE_ARG_OP(op) GENERAL_OP(op, 0, 1, 0)
5697 #define VAR_STORE_OP(op,v) GENERAL_OP(op, v, 1, 0)
5698 #define VAR_STORE_SPEC_OP(op,v) GENERAL_OP(op, v, 1, 1)
5699 #define TWO_ARG_OP(op) GENERAL_OP(op, 0, 2, 1)
5700 #define THREE_ARG_OP(op) GENERAL_OP(op, 0, 3, 2)
5701 // defines for special scale-back-before-store where no token is in context:
5702 #define LOAD_OP_NOTOKEN(op,v,req) do { if (stackSize < req) env->ThrowError("Expr: Not enough elements on stack to perform a load operation"); ops.push_back(ExprOp(op, (v))); maxStackSize = std::max(++stackSize, maxStackSize); } while(0)
5703 #define GENERAL_OP_NOTOKEN(op, v, req, dec) do { if (stackSize < req) env->ThrowError("Expr: Not enough elements on stack to perform an operation"); ops.push_back(ExprOp(op, (v))); stackSize-=(dec); } while(0)
5704 #define TWO_ARG_OP_NOTOKEN(op) GENERAL_OP_NOTOKEN(op, 0, 2, 1)
5705
5706 17 static inline bool isAlphaUnderscore(char c) {
5707
3/10
✗ Branch 2 → 3 not taken.
✓ Branch 2 → 4 taken 17 times.
✗ Branch 3 → 4 not taken.
✗ Branch 3 → 7 not taken.
✗ Branch 4 → 5 not taken.
✓ Branch 4 → 6 taken 17 times.
✗ Branch 5 → 6 not taken.
✗ Branch 5 → 7 not taken.
✗ Branch 6 → 7 not taken.
✓ Branch 6 → 8 taken 17 times.
 17 return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == '_';
5708 }
5709
5710 ✗ static inline bool isAlphaNumUnderscore(char c) {
5711 ✗ return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_';
5712 }
5713
5714 17 static bool isValidVarName(const std::string& s) {
5715 17 size_t len = s.length();
5716
1/2
✗ Branch 3 → 4 not taken.
✓ Branch 3 → 5 taken 17 times.
 17 if (!len)
5717 ✗ return false;
5718
5719
1/2
✓ Branch 7 → 8 taken 17 times.
✗ Branch 7 → 9 not taken.
 17 if (!isAlphaUnderscore(s[0]))
5720 17 return false;
5721 ✗ for (size_t i = 1; i < len; i++)
5722 ✗ if (!isAlphaNumUnderscore(s[i]))
5723 ✗ return false;
5724 ✗ return true;
5725 }
5726
5727
5728 // finds _X suffix (clip letter) and returns 0..25 for x,y,z,a,b,...w
5729 // no suffix means 0
5730 ✗ static int getSuffix(std::string token, std::string base) {
5731 ✗ size_t len = base.length();
5732
5733 ✗ if (token.substr(0, len) != base)
5734 ✗ return -1; // no match
5735
5736 ✗ if (token.length() == len)
5737 ✗ return 0; // no suffix, treat as _x
5738
5739 // find _X suffix, where X is x,y,z,a..w
5740 ✗ if (token.length() != len + 2 || token[len] != '_')
5741 ✗ return -2; // no proper suffix
5742
5743 ✗ char srcChar = token[len + 1]; // last char
5744 int loadIndex;
5745 ✗ if (srcChar >= 'x')
5746 ✗ loadIndex = srcChar - 'x';
5747 else
5748 ✗ loadIndex = srcChar - 'a' + 3;
5749 ✗ return loadIndex;
5750 }
5751
5752 // if automatic source bit depth conversion takes place, ymax, ymin, range_xxx, etc.. constants are changed accordingly
5753 ✗ static int getEffectiveBitsPerComponent(int bitsPerComponent, bool autoconv_conv_float, bool autoconv_conv_int, int autoScaleSourceBitDepth)
5754 {
5755 ✗ if ((autoconv_conv_float && bitsPerComponent == 32) || (autoconv_conv_int && bitsPerComponent != 32))
5756 ✗ return autoScaleSourceBitDepth;
5757 ✗ return bitsPerComponent;
5758 }
5759
5760 41 static size_t parseExpression(const std::string &expr, std::vector<ExprOp> &ops, std::vector<ExprFramePropData>& fp, const VideoInfo **vi, const VideoInfo *vi_output, const SOperation storeOp, int numInputs, int planewidth, int planeheight, bool chroma,
5761 const bool autoconv_full_scale, const bool autoconv_conv_int, const bool autoconv_conv_float, const int clamp_float_i, const bool shift_float, const int lutmode,
5762 IScriptEnvironment *env)
5763 {
5764 // vi_output is new in avs+, and is not used yet
5765
5766 // optional scaling, scale_from bit depth, default scale_to bitdepth, used in scaleb and scalef (yscaleb, yscalef)
5767
1/2
✓ Branch 2 → 3 taken 41 times.
✗ Branch 2 → 2122 not taken.
 41 int targetBitDepth = vi[0]->BitsPerComponent(); // avs+
5768 41 int autoScaleSourceBitDepth = 8; // avs+ scalable constants are in 8 bit range by default
5769
5770 41 std::vector<std::string> tokens;
5771
2/4
✓ Branch 6 → 7 taken 41 times.
✗ Branch 6 → 1749 not taken.
✓ Branch 7 → 8 taken 41 times.
✗ Branch 7 → 1747 not taken.
 41 split(tokens, expr, " \r\n\t", split1::no_empties);
5772
5773 41 std::unordered_map<std::string, int> varnames;
5774 41 int varindex = 0;
5775 41 std::unordered_map<std::string, int> fpnames;
5776 41 int fpindex = 0;
5777
5778 41 size_t maxStackSize = 0;
5779 41 size_t stackSize = 0;
5780
5781
2/2
✓ Branch 1594 → 13 taken 101 times.
✓ Branch 1594 → 1595 taken 41 times.
 142 for (size_t i = 0; i < tokens.size(); i++) {
5782 101 const size_t tokenlen = tokens[i].length();
5783
3/4
✓ Branch 16 → 17 taken 101 times.
✗ Branch 16 → 2116 not taken.
✓ Branch 17 → 18 taken 32 times.
✓ Branch 17 → 25 taken 69 times.
 101 if (tokens[i] == "+")
5784
2/6
✗ Branch 18 → 19 not taken.
✓ Branch 18 → 22 taken 32 times.
✗ Branch 21 → 22 not taken.
✗ Branch 21 → 2116 not taken.
✓ Branch 23 → 24 taken 32 times.
✗ Branch 23 → 1753 not taken.
 32 TWO_ARG_OP(opAdd);
5785
3/4
✓ Branch 26 → 27 taken 69 times.
✗ Branch 26 → 2116 not taken.
✓ Branch 27 → 28 taken 1 time.
✓ Branch 27 → 35 taken 68 times.
 69 else if (tokens[i] == "-")
5786
2/6
✗ Branch 28 → 29 not taken.
✓ Branch 28 → 32 taken 1 time.
✗ Branch 31 → 32 not taken.
✗ Branch 31 → 2116 not taken.
✓ Branch 33 → 34 taken 1 time.
✗ Branch 33 → 1754 not taken.
 1 TWO_ARG_OP(opSub);
5787
2/4
✓ Branch 36 → 37 taken 68 times.
✗ Branch 36 → 2116 not taken.
✗ Branch 37 → 38 not taken.
✓ Branch 37 → 45 taken 68 times.
 68 else if (tokens[i] == "*")
5788 ✗ TWO_ARG_OP(opMul);
5789
2/4
✓ Branch 46 → 47 taken 68 times.
✗ Branch 46 → 2116 not taken.
✗ Branch 47 → 48 not taken.
✓ Branch 47 → 55 taken 68 times.
 68 else if (tokens[i] == "/")
5790 ✗ TWO_ARG_OP(opDiv);
5791
2/4
✓ Branch 56 → 57 taken 68 times.
✗ Branch 56 → 2116 not taken.
✗ Branch 57 → 58 not taken.
✓ Branch 57 → 65 taken 68 times.
 68 else if (tokens[i] == "%")
5792 ✗ TWO_ARG_OP(opFmod);
5793
2/4
✓ Branch 66 → 67 taken 68 times.
✗ Branch 66 → 2116 not taken.
✗ Branch 67 → 68 not taken.
✓ Branch 67 → 75 taken 68 times.
 68 else if (tokens[i] == "max")
5794 ✗ TWO_ARG_OP(opMax);
5795
2/4
✓ Branch 76 → 77 taken 68 times.
✗ Branch 76 → 2116 not taken.
✗ Branch 77 → 78 not taken.
✓ Branch 77 → 85 taken 68 times.
 68 else if (tokens[i] == "min")
5796 ✗ TWO_ARG_OP(opMin);
5797
2/4
✓ Branch 86 → 87 taken 68 times.
✗ Branch 86 → 2116 not taken.
✗ Branch 87 → 88 not taken.
✓ Branch 87 → 95 taken 68 times.
 68 else if (tokens[i] == "exp")
5798 ✗ ONE_ARG_OP(opExp);
5799
2/4
✓ Branch 96 → 97 taken 68 times.
✗ Branch 96 → 2116 not taken.
✗ Branch 97 → 98 not taken.
✓ Branch 97 → 105 taken 68 times.
 68 else if (tokens[i] == "log")
5800 ✗ ONE_ARG_OP(opLog);
5801
5/10
✓ Branch 106 → 107 taken 68 times.
✗ Branch 106 → 2116 not taken.
✓ Branch 107 → 108 taken 68 times.
✗ Branch 107 → 111 not taken.
✓ Branch 109 → 110 taken 68 times.
✗ Branch 109 → 2116 not taken.
✗ Branch 110 → 111 not taken.
✓ Branch 110 → 112 taken 68 times.
✗ Branch 113 → 114 not taken.
✓ Branch 113 → 121 taken 68 times.
 68 else if (tokens[i] == "pow" || tokens[i] == "^") // avs+: ^ can be used for power
5802 ✗ TWO_ARG_OP(opPow);
5803
3/4
✓ Branch 122 → 123 taken 68 times.
✗ Branch 122 → 2116 not taken.
✓ Branch 123 → 124 taken 2 times.
✓ Branch 123 → 131 taken 66 times.
 68 else if (tokens[i] == "sqrt")
5804
2/6
✗ Branch 124 → 125 not taken.
✓ Branch 124 → 128 taken 2 times.
✗ Branch 127 → 128 not taken.
✗ Branch 127 → 2116 not taken.
✓ Branch 129 → 130 taken 2 times.
✗ Branch 129 → 1763 not taken.
 2 ONE_ARG_OP(opSqrt);
5805
2/4
✓ Branch 132 → 133 taken 66 times.
✗ Branch 132 → 2116 not taken.
✗ Branch 133 → 134 not taken.
✓ Branch 133 → 141 taken 66 times.
 66 else if (tokens[i] == "abs")
5806 ✗ ONE_ARG_OP(opAbs);
5807
2/4
✓ Branch 142 → 143 taken 66 times.
✗ Branch 142 → 2116 not taken.
✗ Branch 143 → 144 not taken.
✓ Branch 143 → 151 taken 66 times.
 66 else if (tokens[i] == "sgn")
5808 ✗ ONE_ARG_OP(opSgn);
5809
2/4
✓ Branch 152 → 153 taken 66 times.
✗ Branch 152 → 2116 not taken.
✗ Branch 153 → 154 not taken.
✓ Branch 153 → 161 taken 66 times.
 66 else if (tokens[i] == "sin")
5810 ✗ ONE_ARG_OP(opSin);
5811
2/4
✓ Branch 162 → 163 taken 66 times.
✗ Branch 162 → 2116 not taken.
✗ Branch 163 → 164 not taken.
✓ Branch 163 → 171 taken 66 times.
 66 else if (tokens[i] == "cos")
5812 ✗ ONE_ARG_OP(opCos);
5813
2/4
✓ Branch 172 → 173 taken 66 times.
✗ Branch 172 → 2116 not taken.
✗ Branch 173 → 174 not taken.
✓ Branch 173 → 181 taken 66 times.
 66 else if (tokens[i] == "tan")
5814 ✗ ONE_ARG_OP(opTan);
5815
2/4
✓ Branch 182 → 183 taken 66 times.
✗ Branch 182 → 2116 not taken.
✗ Branch 183 → 184 not taken.
✓ Branch 183 → 191 taken 66 times.
 66 else if (tokens[i] == "asin")
5816 ✗ ONE_ARG_OP(opAsin);
5817
2/4
✓ Branch 192 → 193 taken 66 times.
✗ Branch 192 → 2116 not taken.
✗ Branch 193 → 194 not taken.
✓ Branch 193 → 201 taken 66 times.
 66 else if (tokens[i] == "acos")
5818 ✗ ONE_ARG_OP(opAcos);
5819
2/4
✓ Branch 202 → 203 taken 66 times.
✗ Branch 202 → 2116 not taken.
✗ Branch 203 → 204 not taken.
✓ Branch 203 → 211 taken 66 times.
 66 else if (tokens[i] == "atan")
5820 ✗ ONE_ARG_OP(opAtan);
5821
2/4
✓ Branch 212 → 213 taken 66 times.
✗ Branch 212 → 2116 not taken.
✗ Branch 213 → 214 not taken.
✓ Branch 213 → 221 taken 66 times.
 66 else if (tokens[i] == "atan2")
5822 ✗ TWO_ARG_OP(opAtan2);
5823
2/4
✓ Branch 222 → 223 taken 66 times.
✗ Branch 222 → 2116 not taken.
✗ Branch 223 → 224 not taken.
✓ Branch 223 → 231 taken 66 times.
 66 else if (tokens[i] == "clip")
5824 ✗ THREE_ARG_OP(opClip);
5825
2/4
✓ Branch 232 → 233 taken 66 times.
✗ Branch 232 → 2116 not taken.
✗ Branch 233 → 234 not taken.
✓ Branch 233 → 241 taken 66 times.
 66 else if (tokens[i] == "round")
5826 ✗ ONE_ARG_OP(opRound);
5827
2/4
✓ Branch 242 → 243 taken 66 times.
✗ Branch 242 → 2116 not taken.
✗ Branch 243 → 244 not taken.
✓ Branch 243 → 251 taken 66 times.
 66 else if (tokens[i] == "floor")
5828 ✗ ONE_ARG_OP(opFloor);
5829
2/4
✓ Branch 252 → 253 taken 66 times.
✗ Branch 252 → 2116 not taken.
✗ Branch 253 → 254 not taken.
✓ Branch 253 → 261 taken 66 times.
 66 else if (tokens[i] == "ceil")
5830 ✗ ONE_ARG_OP(opCeil);
5831
2/4
✓ Branch 262 → 263 taken 66 times.
✗ Branch 262 → 2116 not taken.
✗ Branch 263 → 264 not taken.
✓ Branch 263 → 271 taken 66 times.
 66 else if (tokens[i] == "trunc")
5832 ✗ ONE_ARG_OP(opTrunc);
5833
2/4
✓ Branch 272 → 273 taken 66 times.
✗ Branch 272 → 2116 not taken.
✗ Branch 273 → 274 not taken.
✓ Branch 273 → 281 taken 66 times.
 66 else if (tokens[i] == ">")
5834 ✗ TWO_ARG_OP(opGt);
5835
2/4
✓ Branch 282 → 283 taken 66 times.
✗ Branch 282 → 2116 not taken.
✗ Branch 283 → 284 not taken.
✓ Branch 283 → 291 taken 66 times.
 66 else if (tokens[i] == "<")
5836 ✗ TWO_ARG_OP(opLt);
5837
5/10
✓ Branch 292 → 293 taken 66 times.
✗ Branch 292 → 2116 not taken.
✓ Branch 293 → 294 taken 66 times.
✗ Branch 293 → 297 not taken.
✓ Branch 295 → 296 taken 66 times.
✗ Branch 295 → 2116 not taken.
✗ Branch 296 → 297 not taken.
✓ Branch 296 → 298 taken 66 times.
✗ Branch 299 → 300 not taken.
✓ Branch 299 → 307 taken 66 times.
 66 else if (tokens[i] == "=" || tokens[i] == "==") // avs+: == can be used to equality check
5838 ✗ TWO_ARG_OP(opEq);
5839
2/4
✓ Branch 308 → 309 taken 66 times.
✗ Branch 308 → 2116 not taken.
✗ Branch 309 → 310 not taken.
✓ Branch 309 → 317 taken 66 times.
 66 else if (tokens[i] == "!=") // avs+: not equal
5840 ✗ TWO_ARG_OP(opNotEq);
5841
2/4
✓ Branch 318 → 319 taken 66 times.
✗ Branch 318 → 2116 not taken.
✗ Branch 319 → 320 not taken.
✓ Branch 319 → 327 taken 66 times.
 66 else if (tokens[i] == ">=")
5842 ✗ TWO_ARG_OP(opGE);
5843
2/4
✓ Branch 328 → 329 taken 66 times.
✗ Branch 328 → 2116 not taken.
✗ Branch 329 → 330 not taken.
✓ Branch 329 → 337 taken 66 times.
 66 else if (tokens[i] == "<=")
5844 ✗ TWO_ARG_OP(opLE);
5845
2/4
✓ Branch 338 → 339 taken 66 times.
✗ Branch 338 → 2116 not taken.
✗ Branch 339 → 340 not taken.
✓ Branch 339 → 347 taken 66 times.
 66 else if (tokens[i] == "?")
5846 ✗ THREE_ARG_OP(opTernary);
5847
5/10
✓ Branch 348 → 349 taken 66 times.
✗ Branch 348 → 2116 not taken.
✓ Branch 349 → 350 taken 66 times.
✗ Branch 349 → 353 not taken.
✓ Branch 351 → 352 taken 66 times.
✗ Branch 351 → 2116 not taken.
✗ Branch 352 → 353 not taken.
✓ Branch 352 → 354 taken 66 times.
✗ Branch 355 → 356 not taken.
✓ Branch 355 → 363 taken 66 times.
 66 else if (tokens[i] == "and" || tokens[i] == "&") // avs+: & alias for and
5848 ✗ TWO_ARG_OP(opAnd);
5849
5/10
✓ Branch 364 → 365 taken 66 times.
✗ Branch 364 → 2116 not taken.
✓ Branch 365 → 366 taken 66 times.
✗ Branch 365 → 369 not taken.
✓ Branch 367 → 368 taken 66 times.
✗ Branch 367 → 2116 not taken.
✗ Branch 368 → 369 not taken.
✓ Branch 368 → 370 taken 66 times.
✗ Branch 371 → 372 not taken.
✓ Branch 371 → 379 taken 66 times.
 66 else if (tokens[i] == "or" || tokens[i] == "|") // avs+: | alias for or
5850 ✗ TWO_ARG_OP(opOr);
5851
2/4
✓ Branch 380 → 381 taken 66 times.
✗ Branch 380 → 2116 not taken.
✗ Branch 381 → 382 not taken.
✓ Branch 381 → 389 taken 66 times.
 66 else if (tokens[i] == "xor")
5852 ✗ TWO_ARG_OP(opXor);
5853
2/4
✓ Branch 390 → 391 taken 66 times.
✗ Branch 390 → 2116 not taken.
✗ Branch 391 → 392 not taken.
✓ Branch 391 → 399 taken 66 times.
 66 else if (tokens[i] == "not")
5854 ✗ ONE_ARG_OP(opNeg);
5855
2/4
✓ Branch 400 → 401 taken 66 times.
✗ Branch 400 → 2116 not taken.
✗ Branch 401 → 402 not taken.
✓ Branch 401 → 409 taken 66 times.
 66 else if (tokens[i] == "neg")
5856 ✗ ONE_ARG_OP(opNegSign);
5857
3/6
✓ Branch 410 → 411 taken 66 times.
✗ Branch 410 → 1792 not taken.
✓ Branch 411 → 412 taken 66 times.
✗ Branch 411 → 1790 not taken.
✗ Branch 413 → 414 not taken.
✓ Branch 413 → 441 taken 66 times.
 66 else if (tokens[i].substr(0, 3) == "dup")
5858 ✗ if (tokens[i].size() == 3) {
5859 ✗ LOAD_OP(opDup, 0, 1);
5860 }
5861 else {
5862 try {
5863 ✗ int tmp = std::stoi(tokens[i].substr(3));
5864 ✗ if (tmp < 0)
5865 ✗ env->ThrowError("Expr: Dup suffix can't be less than 0 '%s'", tokens[i].c_str());
5866 ✗ LOAD_OP(opDup, tmp, (size_t)(tmp + 1));
5867 }
5868 ✗ catch (std::logic_error &) {
5869 ✗ env->ThrowError("Expr: Failed to convert dup suffix '%s' to valid index", tokens[i].c_str());
5870 ✗ }
5871 }
5872
3/6
✓ Branch 442 → 443 taken 66 times.
✗ Branch 442 → 1809 not taken.
✓ Branch 443 → 444 taken 66 times.
✗ Branch 443 → 1807 not taken.
✗ Branch 445 → 446 not taken.
✓ Branch 445 → 471 taken 66 times.
 66 else if (tokens[i].substr(0, 4) == "swap")
5873 ✗ if (tokens[i].size() == 4) {
5874 ✗ GENERAL_OP(opSwap, 1, 2, 0);
5875 }
5876 else {
5877 try {
5878 ✗ int tmp = std::stoi(tokens[i].substr(4));
5879 ✗ if (tmp < 1)
5880 ✗ env->ThrowError("Expr: Swap suffix can't be less than 1 '%s'", tokens[i].c_str());
5881 ✗ GENERAL_OP(opSwap, tmp, (size_t)(tmp + 1), 0);
5882 }
5883 ✗ catch (std::logic_error &) {
5884 ✗ env->ThrowError("Expr: Failed to convert swap suffix '%s' to valid index", tokens[i].c_str());
5885 ✗ }
5886 }
5887
3/4
✓ Branch 472 → 473 taken 66 times.
✗ Branch 472 → 2116 not taken.
✓ Branch 473 → 474 taken 8 times.
✓ Branch 473 → 480 taken 58 times.
 66 else if (tokens[i] == "sx") { // avs+
5888 // spatial
5889
1/2
✗ Branch 474 → 475 not taken.
✓ Branch 474 → 476 taken 8 times.
 8 if (lutmode > 0)
5890 ✗ env->ThrowError("Expr: 'sx' is forbidden in lut mode");
5891
1/2
✓ Branch 477 → 478 taken 8 times.
✗ Branch 477 → 1824 not taken.
 8 LOAD_OP(opLoadSpatialX, 0, 0);
5892 }
5893
3/4
✓ Branch 481 → 482 taken 58 times.
✗ Branch 481 → 2116 not taken.
✓ Branch 482 → 483 taken 8 times.
✓ Branch 482 → 489 taken 50 times.
 58 else if (tokens[i] == "sy") { // avs+
5894 // spatial
5895
1/2
✗ Branch 483 → 484 not taken.
✓ Branch 483 → 485 taken 8 times.
 8 if (lutmode > 0)
5896 ✗ env->ThrowError("Expr: 'sy' is forbidden in lut mode");
5897
1/2
✓ Branch 486 → 487 taken 8 times.
✗ Branch 486 → 1825 not taken.
 8 LOAD_OP(opLoadSpatialY, 0, 0);
5898 }
5899
2/4
✓ Branch 490 → 491 taken 50 times.
✗ Branch 490 → 2116 not taken.
✗ Branch 491 → 492 not taken.
✓ Branch 491 → 510 taken 50 times.
 50 else if (tokens[i] == "sxr") { // avs+
5900 // spatial X relative 0..1
5901 ✗ if (lutmode > 0)
5902 ✗ env->ThrowError("Expr: 'sxr' is forbidden in lut mode");
5903 ✗ LOAD_OP(opLoadSpatialX, 0, 0);
5904 /* Paranoia: precision at rightmost position? Ensure that sxr == 1.0 there
5905 Multiply by 1/x is different.
5906 Test:
5907 constexpr float x = 1919.0f;
5908 constexpr float y = 1 / 1919.0f;
5909 constexpr float zz = x * y;
5910 constexpr bool b = zz == 1.0f; // false!
5911
5912 const float p = planewidth > 1 ? 1.0f / ((float)planewidth - 1.0f) : 1.0f;
5913 LOAD_OP(opLoadConst, p, 0);
5914 TWO_ARG_OP(opMul);
5915 */
5916 ✗ const float p = planewidth > 1 ? (float)planewidth - 1.0f : 1.0f;
5917 ✗ LOAD_OP(opLoadConst, p, 0);
5918 ✗ TWO_ARG_OP(opDiv);
5919 }
5920
2/4
✓ Branch 511 → 512 taken 50 times.
✗ Branch 511 → 2116 not taken.
✗ Branch 512 → 513 not taken.
✓ Branch 512 → 531 taken 50 times.
 50 else if (tokens[i] == "syr") { // avs+
5921 // spatial Y relative 0..1
5922 ✗ if (lutmode > 0)
5923 ✗ env->ThrowError("Expr: 'syr' is forbidden in lut mode");
5924 ✗ LOAD_OP(opLoadSpatialY, 0, 0);
5925 /* Multiply by 1/x is different
5926 const float p = planeheight > 1 ? 1.0f / ((float)planeheight - 1.0f) : 1.0f;
5927 LOAD_OP(opLoadConst, p, 0);
5928 TWO_ARG_OP(opMul);
5929 */
5930 ✗ const float p = planeheight > 1 ? (float)planeheight - 1.0f : 1.0f;
5931 ✗ LOAD_OP(opLoadConst, p, 0);
5932 ✗ TWO_ARG_OP(opDiv);
5933 }
5934
2/4
✓ Branch 532 → 533 taken 50 times.
✗ Branch 532 → 2116 not taken.
✗ Branch 533 → 534 not taken.
✓ Branch 533 → 540 taken 50 times.
 50 else if (tokens[i] == "frameno") { // avs+
5935 ✗ if (lutmode > 0)
5936 ✗ env->ThrowError("Expr: 'frameno' is forbidden in lut mode");
5937 ✗ LOAD_OP(opLoadInternalVar, INTERNAL_VAR_CURRENT_FRAME, 0);
5938 }
5939
2/4
✓ Branch 541 → 542 taken 50 times.
✗ Branch 541 → 2116 not taken.
✗ Branch 542 → 543 not taken.
✓ Branch 542 → 549 taken 50 times.
 50 else if (tokens[i] == "time") { // avs+
5940 ✗ if (lutmode > 0)
5941 ✗ env->ThrowError("Expr: 'time' is forbidden in lut mode");
5942 ✗ LOAD_OP(opLoadInternalVar, INTERNAL_VAR_RELTIME, 0);
5943 }
5944
2/4
✓ Branch 550 → 551 taken 50 times.
✗ Branch 550 → 2116 not taken.
✗ Branch 551 → 552 not taken.
✓ Branch 551 → 556 taken 50 times.
 50 else if (tokens[i] == "width") { // avs+
5945 ✗ LOAD_OP(opLoadConst, (float)planewidth, 0);
5946 }
5947
2/4
✓ Branch 557 → 558 taken 50 times.
✗ Branch 557 → 2116 not taken.
✗ Branch 558 → 559 not taken.
✓ Branch 558 → 563 taken 50 times.
 50 else if (tokens[i] == "height") { // avs+
5948 ✗ LOAD_OP(opLoadConst, (float)planeheight, 0);
5949 }
5950
13/18
✓ Branch 565 → 566 taken 15 times.
✓ Branch 565 → 572 taken 35 times.
✓ Branch 568 → 569 taken 15 times.
✗ Branch 568 → 579 not taken.
✓ Branch 570 → 571 taken 15 times.
✗ Branch 570 → 2116 not taken.
✓ Branch 571 → 572 taken 2 times.
✓ Branch 571 → 579 taken 13 times.
✓ Branch 573 → 574 taken 37 times.
✗ Branch 573 → 2116 not taken.
✓ Branch 574 → 575 taken 33 times.
✓ Branch 574 → 579 taken 4 times.
✓ Branch 576 → 577 taken 33 times.
✗ Branch 576 → 2116 not taken.
✓ Branch 577 → 578 taken 33 times.
✗ Branch 577 → 579 not taken.
✓ Branch 580 → 581 taken 33 times.
✓ Branch 580 → 775 taken 17 times.
 50 else if ((tokens[i].length() == 1 || (tokens[i].length() > 1 && tokens[i][1] == '[')) && tokens[i][0] >= 'a' && tokens[i][0] <= 'z') {
5951 33 const bool rel = tokens[i].length() > 1; // relative pixel addressing; indexed clips e.g. x[-1,-2]
5952 // loading source clip pixels
5953
1/2
✓ Branch 584 → 585 taken 33 times.
✗ Branch 584 → 2116 not taken.
 33 char srcChar = tokens[i][0];
5954 int loadIndex;
5955
1/2
✓ Branch 585 → 586 taken 33 times.
✗ Branch 585 → 587 not taken.
 33 if (srcChar >= 'x')
5956 33 loadIndex = srcChar - 'x';
5957 else
5958 ✗ loadIndex = srcChar - 'a' + 3;
5959
1/2
✗ Branch 588 → 589 not taken.
✓ Branch 588 → 592 taken 33 times.
 33 if (loadIndex >= numInputs)
5960 ✗ env->ThrowError("Expr: Too few input clips supplied to reference '%s'", tokens[i].c_str());
5961
5962
2/2
✓ Branch 592 → 593 taken 2 times.
✓ Branch 592 → 648 taken 31 times.
 33 if (rel) {
5963 int dx, dy;
5964 2 std::string s;
5965
2/4
✓ Branch 595 → 596 taken 2 times.
✗ Branch 595 → 1838 not taken.
✓ Branch 596 → 597 taken 2 times.
✗ Branch 596 → 1836 not taken.
 2 std::istringstream numStream(tokens[i].substr(2)); // after '['
5966
2/4
✓ Branch 598 → 599 taken 2 times.
✗ Branch 598 → 1839 not taken.
✓ Branch 599 → 600 taken 2 times.
✗ Branch 599 → 1839 not taken.
 2 numStream.imbue(std::locale::classic());
5967 // first coord
5968
3/6
✓ Branch 601 → 602 taken 2 times.
✗ Branch 601 → 1841 not taken.
✓ Branch 602 → 603 taken 2 times.
✗ Branch 602 → 1841 not taken.
✗ Branch 603 → 604 not taken.
✓ Branch 603 → 607 taken 2 times.
 2 if (!(numStream >> dx))
5969 ✗ env->ThrowError("Expr: Failed to convert '%s' to integer, relative index dx", tokens[i].c_str());
5970 // separator ','
5971
2/4
✓ Branch 607 → 608 taken 2 times.
✗ Branch 607 → 1841 not taken.
✗ Branch 608 → 609 not taken.
✓ Branch 608 → 612 taken 2 times.
 2 if (numStream.get() != ',')
5972 ✗ env->ThrowError("Expr: Failed to convert '%s', character ',' expected between the coordinates", tokens[i].c_str());
5973 // second coord
5974
3/6
✓ Branch 612 → 613 taken 2 times.
✗ Branch 612 → 1841 not taken.
✓ Branch 613 → 614 taken 2 times.
✗ Branch 613 → 1841 not taken.
✗ Branch 614 → 615 not taken.
✓ Branch 614 → 618 taken 2 times.
 2 if (!(numStream >> dy))
5975 ✗ env->ThrowError("Expr: Failed to convert '%s' to integer, relative index dy", tokens[i].c_str());
5976 // ending ']'
5977
2/4
✓ Branch 618 → 619 taken 2 times.
✗ Branch 618 → 1841 not taken.
✗ Branch 619 → 620 not taken.
✓ Branch 619 → 623 taken 2 times.
 2 if (numStream.get() != ']')
5978 ✗ env->ThrowError("Expr: Failed to convert '%s' to [x,y], closing ']' expected ", tokens[i].c_str());
5979
3/6
✓ Branch 623 → 624 taken 2 times.
✗ Branch 623 → 1841 not taken.
✓ Branch 624 → 625 taken 2 times.
✗ Branch 624 → 1841 not taken.
✗ Branch 625 → 626 not taken.
✓ Branch 625 → 629 taken 2 times.
 2 if (numStream >> s)
5980 ✗ env->ThrowError("Expr: Failed to convert '%s' to [x,y], invalid character after ']'", tokens[i].c_str());
5981
5982
1/2
✗ Branch 629 → 630 not taken.
✓ Branch 629 → 631 taken 2 times.
 2 if (lutmode > 0)
5983 ✗ env->ThrowError("Expr: relative pixel addressing is forbidden in lut mode");
5984
5985
2/4
✓ Branch 631 → 632 taken 2 times.
✗ Branch 631 → 633 not taken.
✗ Branch 632 → 633 not taken.
✓ Branch 632 → 636 taken 2 times.
 2 if (dx <= -vi_output->width || dx >= vi_output->width)
5986 ✗ env->ThrowError("Expr: dx must be between +/- (width-1) in '%s'", tokens[i].c_str());
5987
2/4
✓ Branch 636 → 637 taken 2 times.
✗ Branch 636 → 638 not taken.
✗ Branch 637 → 638 not taken.
✓ Branch 637 → 641 taken 2 times.
 2 if (dy <= -vi_output->height || dy >= vi_output->height)
5988 ✗ env->ThrowError("Expr: dy must be between +/- (height-1) in '%s'", tokens[i].c_str());
5989
2/4
✓ Branch 641 → 642 taken 2 times.
✗ Branch 641 → 1840 not taken.
✓ Branch 643 → 644 taken 2 times.
✗ Branch 643 → 1840 not taken.
 2 LOAD_REL_OP(getLoadOp(vi[loadIndex], true), loadIndex, 0, dx, dy);
5990 2 }
5991 else {
5992 // not relative, single clip letter
5993
2/2
✓ Branch 648 → 649 taken 30 times.
✓ Branch 648 → 654 taken 1 time.
 31 if (lutmode == 0) {
5994
2/4
✓ Branch 649 → 650 taken 30 times.
✗ Branch 649 → 1846 not taken.
✓ Branch 651 → 652 taken 30 times.
✗ Branch 651 → 1846 not taken.
 30 LOAD_OP(getLoadOp(vi[loadIndex], false), loadIndex, 0);
5995 }
5996 else {
5997 // for lut we replace x and y to sx and sy to make the initialization
5998
1/2
✗ Branch 654 → 655 not taken.
✓ Branch 654 → 658 taken 1 time.
 1 if (loadIndex >= lutmode) // lutx
5999 ✗ env->ThrowError("Expr: more input clips than lut's dimension. Problematic clip: '%s'", tokens[i].c_str());
6000 // spatial
6001
1/2
✓ Branch 658 → 659 taken 1 time.
✗ Branch 658 → 663 not taken.
 1 if (loadIndex == 0)
6002
1/2
✓ Branch 660 → 661 taken 1 time.
✗ Branch 660 → 1847 not taken.
 1 LOAD_OP(opLoadSpatialX, 0, 0);
6003 else // if (loadIndex == 1)
6004 ✗ LOAD_OP(opLoadSpatialY, 0, 0);
6005 }
6006 }
6007
6008 // avs+: 'scale_inputs': converts input pixels to a common specified range
6009 // Apply full or limited conversion to integer and/or float bit depths.
6010 // For integers bit-shift or full-scale-stretch method can be chosen
6011 // There is no precision loss, since the multiplication/division occurs when original pixels
6012 // are already loaded as float
6013
1/2
✓ Branch 667 → 668 taken 33 times.
✗ Branch 667 → 2116 not taken.
 33 const int srcBitDepth = vi[loadIndex]->BitsPerComponent();
6014 33 const int dstBitDepth = autoScaleSourceBitDepth; // internal precision
6015
6016 33 const bool use_chroma = chroma; // && !forceNonUV;
6017 33 const bool isfull = autoconv_full_scale;
6018
6019
2/4
✓ Branch 668 → 669 taken 33 times.
✗ Branch 668 → 670 not taken.
✗ Branch 669 → 670 not taken.
✓ Branch 669 → 760 taken 33 times.
 33 if (autoconv_conv_int || autoconv_conv_float) {
6020
6021 ✗ if ((srcBitDepth != 32 && autoconv_conv_int) ||
6022 ✗ ((srcBitDepth == 32 && autoconv_conv_float))) {
6023
6024 ✗ bits_conv_constants d;
6025 ✗ get_bits_conv_constants(d, use_chroma, isfull, isfull, srcBitDepth, dstBitDepth);
6026 // chroma is spec: signed. Limited:16-240 is really 128 +/-112. Full:1-255 is really 128+/-127
6027
6028 ✗ if (d.src_offset != 0) {
6029 ✗ LOAD_OP(opLoadConst, (float)d.src_offset, 0);
6030 ✗ TWO_ARG_OP(opSub);
6031 }
6032 ✗ if (d.mul_factor != 1.0f) {
6033 ✗ LOAD_OP(opLoadConst, d.mul_factor, 0);
6034 ✗ TWO_ARG_OP(opMul);
6035 }
6036 ✗ if (d.dst_offset != 0) {
6037 ✗ LOAD_OP(opLoadConst, (float)d.dst_offset, 0);
6038 ✗ TWO_ARG_OP(opAdd);
6039 }
6040 }
6041 }
6042
6043 // "floatUV" - shifts -0.5 .. +0.5 chroma range to 0 .. 1.0 only when no other autoscaling is active
6044 // the effect of this pre-shift is reversed at the storage phase
6045
3/6
✗ Branch 760 → 761 not taken.
✓ Branch 760 → 762 taken 33 times.
✗ Branch 761 → 762 not taken.
✗ Branch 761 → 1592 not taken.
✓ Branch 762 → 763 taken 2 times.
✓ Branch 762 → 1592 taken 31 times.
 33 if ((!autoconv_conv_float || dstBitDepth == 32) && srcBitDepth == 32) {
6046
1/4
✗ Branch 763 → 764 not taken.
✓ Branch 763 → 1592 taken 2 times.
✗ Branch 764 → 765 not taken.
✗ Branch 764 → 1592 not taken.
 2 if (chroma && shift_float) {
6047 ✗ LOAD_OP(opLoadConst, 0.5f, 0);
6048 ✗ TWO_ARG_OP(opAdd); // at the end pixel exit: opSub
6049 }
6050 }
6051 }
6052
2/4
✓ Branch 776 → 777 taken 17 times.
✗ Branch 776 → 2116 not taken.
✗ Branch 777 → 778 not taken.
✓ Branch 777 → 783 taken 17 times.
 17 else if (tokens[i] == "pi") // avs+
6053 {
6054 ✗ float pi = 3.141592653589793f;
6055 ✗ LOAD_OP(opLoadConst, pi, 0);
6056 }
6057 // avs+
6058 // bitdepth: automatic silent parameter of the lut expression(clip bit depth)
6059 // clip-specific bitdepths: bitdepth_x, bitdepth_y, bitdepth_z, bitdepth_a, .. bitdepth_w,
6060 // sbitdepth : automatic silent parameter of the lut expression(bit depth of values to scale)
6061 //
6062 // pre-defined, bit depth aware constants
6063 // range_half : autoscaled 128 or 0.5 for float, (or 0.0 for chroma with zero-base float chroma version)
6064 // range_max : 255 / 1023 / 4095 / 16383 / 65535 or 1.0 for float
6065 // 0.5 for float chroma (new zero-based style)
6066 // range_min : 0 for 8-16bits, or 0 for float luma, or -0.5 for float chroma
6067 // -0.5 for float chroma (new zero-based style)
6068 // range_size : 256 / 1024...65536
6069 // ymin, ymax : 16 / 235 autoscaled.
6070 // cmin, cmax : 16 / 240 autoscaled. For 32bits zero based chroma: (16-128)/255.0, (240-128)/255.0
6071
6072 // ymin or ymin_x, ymin_y, ymin_y, ymin_a....
6073 // similarly: ymax, range_max, cmin, cmax, range_half
6074 // without clip index specifier, or with '_'+letter suffix
6075
3/6
✓ Branch 784 → 785 taken 17 times.
✗ Branch 784 → 1861 not taken.
✓ Branch 785 → 786 taken 17 times.
✗ Branch 785 → 1859 not taken.
✗ Branch 787 → 788 not taken.
✓ Branch 787 → 821 taken 17 times.
 17 else if (tokens[i].substr(0, 8) == "bitdepth") // avs+
6076 {
6077 ✗ int loadIndex = -1;
6078 ✗ std::string toFind = "bitdepth";
6079 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6080 ✗ loadIndex = getSuffix(tokens[i], toFind);
6081 ✗ if (loadIndex < 0)
6082 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6083 ✗ if (loadIndex >= numInputs)
6084 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6085
6086 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6087 ✗ float q = (float)bitsPerComponent;
6088 ✗ LOAD_OP(opLoadConst, q, 0);
6089 ✗ }
6090
2/4
✓ Branch 822 → 823 taken 17 times.
✗ Branch 822 → 2116 not taken.
✗ Branch 823 → 824 not taken.
✓ Branch 823 → 829 taken 17 times.
 17 else if (tokens[i] == "sbitdepth") // avs+
6091 {
6092 ✗ float q = (float)autoScaleSourceBitDepth;
6093 ✗ LOAD_OP(opLoadConst, q, 0);
6094 }
6095
3/6
✓ Branch 830 → 831 taken 17 times.
✗ Branch 830 → 1879 not taken.
✓ Branch 831 → 832 taken 17 times.
✗ Branch 831 → 1877 not taken.
✗ Branch 833 → 834 not taken.
✓ Branch 833 → 869 taken 17 times.
 17 else if (tokens[i].substr(0, 4) == "ymin") // avs+
6096 {
6097 ✗ int loadIndex = -1;
6098 ✗ std::string toFind = "ymin";
6099 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6100 ✗ loadIndex = getSuffix(tokens[i], toFind);
6101 ✗ if (loadIndex < 0)
6102 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6103 ✗ if (loadIndex >= numInputs)
6104 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6105
6106 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6107 ✗ float q = bitsPerComponent == 32 ? 16.0f / 255 : (16 << (bitsPerComponent - 8)); // scale luma min 16
6108 ✗ LOAD_OP(opLoadConst, q, 0);
6109 ✗ }
6110
3/6
✓ Branch 870 → 871 taken 17 times.
✗ Branch 870 → 1896 not taken.
✓ Branch 871 → 872 taken 17 times.
✗ Branch 871 → 1894 not taken.
✗ Branch 873 → 874 not taken.
✓ Branch 873 → 909 taken 17 times.
 17 else if (tokens[i].substr(0, 4) == "ymax") // avs+
6111 {
6112 ✗ int loadIndex = -1;
6113 ✗ std::string toFind = "ymax";
6114 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6115 ✗ loadIndex = getSuffix(tokens[i], toFind);
6116 ✗ if (loadIndex < 0)
6117 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6118 ✗ if (loadIndex >= numInputs)
6119 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6120
6121 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6122 ✗ float q = bitsPerComponent == 32 ? 235.0f / 255 : (235 << (bitsPerComponent - 8)); // scale luma max 235
6123 ✗ LOAD_OP(opLoadConst, q, 0);
6124 ✗ }
6125
3/6
✓ Branch 910 → 911 taken 17 times.
✗ Branch 910 → 1913 not taken.
✓ Branch 911 → 912 taken 17 times.
✗ Branch 911 → 1911 not taken.
✗ Branch 913 → 914 not taken.
✓ Branch 913 → 954 taken 17 times.
 17 else if (tokens[i].substr(0, 4) == "cmin") // avs+
6126 {
6127 ✗ int loadIndex = -1;
6128 ✗ std::string toFind = "cmin";
6129 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6130 ✗ loadIndex = getSuffix(tokens[i], toFind);
6131 ✗ if (loadIndex < 0)
6132 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6133 ✗ if (loadIndex >= numInputs)
6134 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6135
6136 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6137
6138 ✗ float q = bitsPerComponent == 32 ? uv8tof(16) : (16 << (bitsPerComponent - 8)); // scale chroma min 16
6139 ✗ if (shift_float && bitsPerComponent) q += 0.5f;
6140 ✗ LOAD_OP(opLoadConst, q, 0);
6141 ✗ }
6142
3/6
✓ Branch 955 → 956 taken 17 times.
✗ Branch 955 → 1930 not taken.
✓ Branch 956 → 957 taken 17 times.
✗ Branch 956 → 1928 not taken.
✗ Branch 958 → 959 not taken.
✓ Branch 958 → 999 taken 17 times.
 17 else if (tokens[i].substr(0, 4) == "cmax") // avs+
6143 {
6144 ✗ int loadIndex = -1;
6145 ✗ std::string toFind = "cmax";
6146 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6147 ✗ loadIndex = getSuffix(tokens[i], toFind);
6148 ✗ if (loadIndex < 0)
6149 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6150 ✗ if (loadIndex >= numInputs)
6151 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6152
6153 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6154 ✗ float q = bitsPerComponent == 32 ? uv8tof(240) : (240 << (bitsPerComponent - 8)); // scale chroma max 240
6155 ✗ if (shift_float && bitsPerComponent == 32) q += 0.5f;
6156 ✗ LOAD_OP(opLoadConst, q, 0);
6157 ✗ }
6158
3/6
✓ Branch 1000 → 1001 taken 17 times.
✗ Branch 1000 → 1947 not taken.
✓ Branch 1001 → 1002 taken 17 times.
✗ Branch 1001 → 1945 not taken.
✗ Branch 1003 → 1004 not taken.
✓ Branch 1003 → 1039 taken 17 times.
 17 else if (tokens[i].substr(0, 10) == "range_size") // avs+
6159 {
6160 ✗ int loadIndex = -1;
6161 ✗ std::string toFind = "range_size";
6162 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6163 ✗ loadIndex = getSuffix(tokens[i], toFind);
6164 ✗ if (loadIndex < 0)
6165 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6166 ✗ if (loadIndex >= numInputs)
6167 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6168
6169 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6170 ✗ float q = bitsPerComponent == 32 ? 1.0f : (1 << bitsPerComponent); // 1.0, 256, 1024,... 65536
6171 ✗ LOAD_OP(opLoadConst, q, 0);
6172 ✗ }
6173
3/6
✓ Branch 1040 → 1041 taken 17 times.
✗ Branch 1040 → 1964 not taken.
✓ Branch 1041 → 1042 taken 17 times.
✗ Branch 1041 → 1962 not taken.
✗ Branch 1043 → 1044 not taken.
✓ Branch 1043 → 1085 taken 17 times.
 17 else if (tokens[i].substr(0, 9) == "range_min") // avs+ > r2636
6174 {
6175 ✗ int loadIndex = -1;
6176 ✗ std::string toFind = "range_min";
6177 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6178 ✗ loadIndex = getSuffix(tokens[i], toFind);
6179 ✗ if (loadIndex < 0)
6180 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6181 ✗ if (loadIndex >= numInputs)
6182 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6183
6184 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6185 // 0.0 (or -0.5 for 32bit float chroma)
6186 ✗ float q = bitsPerComponent == 32 ? (chroma ? -0.5f : 0.0f) : 0;
6187 ✗ if (chroma && shift_float && bitsPerComponent == 32) q += 0.5f;
6188 ✗ LOAD_OP(opLoadConst, q, 0);
6189 ✗ }
6190
3/6
✓ Branch 1086 → 1087 taken 17 times.
✗ Branch 1086 → 1981 not taken.
✓ Branch 1087 → 1088 taken 17 times.
✗ Branch 1087 → 1979 not taken.
✗ Branch 1089 → 1090 not taken.
✓ Branch 1089 → 1123 taken 17 times.
 17 else if (tokens[i].substr(0, 10) == "yrange_min")
6191 {
6192 ✗ int loadIndex = -1;
6193 ✗ std::string toFind = "yrange_min";
6194 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6195 ✗ loadIndex = getSuffix(tokens[i], toFind);
6196 ✗ if (loadIndex < 0)
6197 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6198 ✗ if (loadIndex >= numInputs)
6199 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6200
6201 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6202 ✗ float q = bitsPerComponent == 32 ? 0.0f : 0;
6203 ✗ LOAD_OP(opLoadConst, q, 0);
6204 ✗ }
6205
3/6
✓ Branch 1124 → 1125 taken 17 times.
✗ Branch 1124 → 1998 not taken.
✓ Branch 1125 → 1126 taken 17 times.
✗ Branch 1125 → 1996 not taken.
✗ Branch 1127 → 1128 not taken.
✓ Branch 1127 → 1169 taken 17 times.
 17 else if (tokens[i].substr(0, 9) == "range_max") // avs+
6206 {
6207 ✗ int loadIndex = -1;
6208 ✗ std::string toFind = "range_max";
6209 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6210 ✗ loadIndex = getSuffix(tokens[i], toFind);
6211 ✗ if (loadIndex < 0)
6212 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6213 ✗ if (loadIndex >= numInputs)
6214 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6215
6216 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6217 // 1.0 (or 0.5 for 32bit float chroma), 255, 1023,... 65535
6218 ✗ float q = bitsPerComponent == 32 ? (chroma ? + 0.5f : 1.0f) : ((1 << bitsPerComponent) - 1);
6219 ✗ if (chroma && shift_float && bitsPerComponent == 32) q += 0.5f;
6220 ✗ LOAD_OP(opLoadConst, q, 0);
6221 ✗ }
6222
3/6
✓ Branch 1170 → 1171 taken 17 times.
✗ Branch 1170 → 2015 not taken.
✓ Branch 1171 → 1172 taken 17 times.
✗ Branch 1171 → 2013 not taken.
✗ Branch 1173 → 1174 not taken.
✓ Branch 1173 → 1209 taken 17 times.
 17 else if (tokens[i].substr(0, 10) == "yrange_max")
6223 {
6224 ✗ int loadIndex = -1;
6225 ✗ std::string toFind = "yrange_max";
6226 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6227 ✗ loadIndex = getSuffix(tokens[i], toFind);
6228 ✗ if (loadIndex < 0)
6229 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6230 ✗ if (loadIndex >= numInputs)
6231 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6232
6233 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6234 // 1.0, 255, 1023,... 65535
6235 ✗ float q = bitsPerComponent == 32 ? 1.0f : ((1 << bitsPerComponent) - 1);
6236 ✗ LOAD_OP(opLoadConst, q, 0);
6237 ✗ }
6238
3/6
✓ Branch 1210 → 1211 taken 17 times.
✗ Branch 1210 → 2032 not taken.
✓ Branch 1211 → 1212 taken 17 times.
✗ Branch 1211 → 2030 not taken.
✗ Branch 1213 → 1214 not taken.
✓ Branch 1213 → 1257 taken 17 times.
 17 else if (tokens[i].substr(0, 10) == "range_half") // avs+
6239 {
6240 ✗ int loadIndex = -1;
6241 ✗ std::string toFind = "range_half";
6242 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6243 ✗ loadIndex = getSuffix(tokens[i], toFind);
6244 ✗ if (loadIndex < 0)
6245 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6246 ✗ if (loadIndex >= numInputs)
6247 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6248
6249 // for chroma: range_half is 0.0 for 32bit float (or 0.5 for old float chroma representation)
6250 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6251 ✗ float q = bitsPerComponent == 32 ? (chroma ? uv8tof(128) : 0.5f) : (1 << (bitsPerComponent - 1)); // 0.5f, 128, 512, ... 32768
6252 ✗ if (chroma && shift_float && bitsPerComponent == 32) q += 0.5f;
6253 ✗ LOAD_OP(opLoadConst, q, 0);
6254 ✗ }
6255
3/6
✓ Branch 1258 → 1259 taken 17 times.
✗ Branch 1258 → 2049 not taken.
✓ Branch 1259 → 1260 taken 17 times.
✗ Branch 1259 → 2047 not taken.
✗ Branch 1261 → 1262 not taken.
✓ Branch 1261 → 1297 taken 17 times.
 17 else if (tokens[i].substr(0, 11) == "yrange_half") // avs+
6256 {
6257 ✗ int loadIndex = -1;
6258 ✗ std::string toFind = "yrange_half";
6259 ✗ if (tokens[i].substr(0, toFind.length()) == toFind)
6260 ✗ loadIndex = getSuffix(tokens[i], toFind);
6261 ✗ if (loadIndex < 0)
6262 ✗ env->ThrowError("Expr: Error in built-in constant expression '%s'", tokens[i].c_str());
6263 ✗ if (loadIndex >= numInputs)
6264 ✗ env->ThrowError("Expr: Too few input clips supplied for reference '%s'", tokens[i].c_str());
6265
6266 ✗ int bitsPerComponent = getEffectiveBitsPerComponent(vi[loadIndex]->BitsPerComponent(), autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6267 ✗ float q = bitsPerComponent == 32 ? 0.5f : (1 << (bitsPerComponent - 1)); // 0.5f, 128, 512, ... 32768
6268 ✗ LOAD_OP(opLoadConst, q, 0);
6269 ✗ }
6270 // "scaleb" and "scalef" functions scale their operand from 8 bit to the bit depth of the first clip.
6271 // "i8", "i10", "i14", "i16" and "f32" (typically at the beginning of the expression) sets the scale-base to 8..16 bits or float, respectively.
6272 // "i8".."f32" keywords can appear anywhere in the expression, but only the last occurence will be effective for the whole expression.
6273
9/18
✓ Branch 1298 → 1299 taken 17 times.
✗ Branch 1298 → 2116 not taken.
✓ Branch 1299 → 1300 taken 17 times.
✗ Branch 1299 → 1309 not taken.
✓ Branch 1301 → 1302 taken 17 times.
✗ Branch 1301 → 2116 not taken.
✓ Branch 1302 → 1303 taken 17 times.
✗ Branch 1302 → 1309 not taken.
✓ Branch 1304 → 1305 taken 17 times.
✗ Branch 1304 → 2116 not taken.
✓ Branch 1305 → 1306 taken 17 times.
✗ Branch 1305 → 1309 not taken.
✓ Branch 1307 → 1308 taken 17 times.
✗ Branch 1307 → 2116 not taken.
✗ Branch 1308 → 1309 not taken.
✓ Branch 1308 → 1310 taken 17 times.
✗ Branch 1311 → 1312 not taken.
✓ Branch 1311 → 1424 taken 17 times.
 17 else if (tokens[i] == "scaleb" || tokens[i] == "yscaleb" || tokens[i] == "scalef" || tokens[i] == "yscalef") // avs+, scale by bit shift
6274 {
6275 // Note: if 'scale_float' is used then all float input is automatically converted to integer
6276 // in this case the targetBitDepth is not 32 for float clips but the actual autoscaleSourceBitDepth
6277 ✗ int effectivetargetBitDepth = getEffectiveBitsPerComponent(targetBitDepth, autoconv_conv_float, autoconv_conv_int, autoScaleSourceBitDepth);
6278 // number to scale is not chroma-related one even if we are in chroma (U,V) plane
6279 ✗ const bool forceNonUV = (tokens[i] == "yscaleb") || (tokens[i] == "yscalef");
6280
6281 ✗ const int srcBitDepth = autoScaleSourceBitDepth;
6282 ✗ const int dstBitDepth = effectivetargetBitDepth;
6283 ✗ const bool use_chroma = chroma && !forceNonUV;
6284
6285 ✗ const bool isfull = tokens[i] == "scalef" || tokens[i] == "yscalef";
6286
6287 ✗ bits_conv_constants d;
6288 ✗ get_bits_conv_constants(d, use_chroma, isfull, isfull, srcBitDepth, dstBitDepth);
6289 // chroma is spec: signed. Limited:16-240 is really 128 +/-112. Full:1-255 is really 128+/-127
6290
6291 // floatUV
6292 ✗ if (use_chroma && shift_float && dstBitDepth == 32 && srcBitDepth < 32) {
6293 ✗ d.dst_offset = 0.5f; // get out from -0.5..0.5 to 0..1
6294 }
6295
6296 ✗ if (d.src_offset != 0) {
6297 ✗ LOAD_OP(opLoadConst, (float)d.src_offset, 0);
6298 ✗ TWO_ARG_OP(opSub);
6299 }
6300 ✗ if (d.mul_factor != 1.0f) {
6301 ✗ LOAD_OP(opLoadConst, d.mul_factor, 0);
6302 ✗ TWO_ARG_OP(opMul);
6303 }
6304 ✗ if (d.dst_offset != 0) {
6305 ✗ LOAD_OP(opLoadConst, (float)d.dst_offset, 0);
6306 ✗ TWO_ARG_OP(opAdd);
6307 }
6308 }
6309
2/4
✓ Branch 1425 → 1426 taken 17 times.
✗ Branch 1425 → 2116 not taken.
✗ Branch 1426 → 1427 not taken.
✓ Branch 1426 → 1428 taken 17 times.
 17 else if (tokens[i] == "i8") // avs+
6310 {
6311 ✗ autoScaleSourceBitDepth = 8;
6312 }
6313
2/4
✓ Branch 1429 → 1430 taken 17 times.
✗ Branch 1429 → 2116 not taken.
✗ Branch 1430 → 1431 not taken.
✓ Branch 1430 → 1432 taken 17 times.
 17 else if (tokens[i] == "i10") // avs+
6314 {
6315 ✗ autoScaleSourceBitDepth = 10;
6316 }
6317
2/4
✓ Branch 1433 → 1434 taken 17 times.
✗ Branch 1433 → 2116 not taken.
✗ Branch 1434 → 1435 not taken.
✓ Branch 1434 → 1436 taken 17 times.
 17 else if (tokens[i] == "i12") // avs+
6318 {
6319 ✗ autoScaleSourceBitDepth = 12;
6320 }
6321
2/4
✓ Branch 1437 → 1438 taken 17 times.
✗ Branch 1437 → 2116 not taken.
✗ Branch 1438 → 1439 not taken.
✓ Branch 1438 → 1440 taken 17 times.
 17 else if (tokens[i] == "i14") // avs+
6322 {
6323 ✗ autoScaleSourceBitDepth = 14;
6324 }
6325
2/4
✓ Branch 1441 → 1442 taken 17 times.
✗ Branch 1441 → 2116 not taken.
✗ Branch 1442 → 1443 not taken.
✓ Branch 1442 → 1444 taken 17 times.
 17 else if (tokens[i] == "i16") // avs+
6326 {
6327 ✗ autoScaleSourceBitDepth = 16;
6328 }
6329
2/4
✓ Branch 1445 → 1446 taken 17 times.
✗ Branch 1445 → 2116 not taken.
✗ Branch 1446 → 1447 not taken.
✓ Branch 1446 → 1448 taken 17 times.
 17 else if (tokens[i] == "f32") // avs+
6330 {
6331 ✗ autoScaleSourceBitDepth = 32;
6332 }
6333
5/16
✓ Branch 1450 → 1451 taken 13 times.
✓ Branch 1450 → 1461 taken 4 times.
✓ Branch 1452 → 1453 taken 13 times.
✗ Branch 1452 → 2116 not taken.
✗ Branch 1453 → 1454 not taken.
✓ Branch 1453 → 1461 taken 13 times.
✗ Branch 1455 → 1456 not taken.
✗ Branch 1455 → 2116 not taken.
✗ Branch 1456 → 1457 not taken.
✗ Branch 1456 → 1461 not taken.
✗ Branch 1458 → 1459 not taken.
✗ Branch 1458 → 2116 not taken.
✗ Branch 1459 → 1460 not taken.
✗ Branch 1459 → 1461 not taken.
✗ Branch 1462 → 1463 not taken.
✓ Branch 1462 → 1505 taken 17 times.
 17 else if (tokens[i].length() > 1 && tokens[i][0] >= 'a' && tokens[i][0] <= 'z' && tokens[i][1] == '.') {
6334 // frame property access: x.framePropName syntax
6335 ✗ char srcChar = tokens[i][0];
6336 int srcIndex;
6337 ✗ if (srcChar >= 'x')
6338 ✗ srcIndex = srcChar - 'x';
6339 else
6340 ✗ srcIndex = srcChar - 'a' + 3;
6341 ✗ if (srcIndex >= numInputs)
6342 ✗ env->ThrowError("Expr: Too few input clips supplied to reference '%s'", tokens[i].c_str());
6343
6344 ✗ auto fullname = tokens[i];
6345 ✗ auto fpname = tokens[i].substr(2); // after '.'
6346 ✗ if(fpname.length() == 0)
6347 ✗ env->ThrowError("Expr: no frame property name is specified");
6348 ✗ if (!isValidVarName(fpname))
6349 ✗ env->ThrowError("Expr: invalid frame property name '%s'", fpname.c_str());
6350
6351 // check if frame property already existed in a variable slots
6352 ✗ auto key = fullname;
6353 ✗ auto it = fpnames.find(key);
6354 int loadIndex;
6355 ✗ if (it == fpnames.end()) {
6356 // first occurance, insert name and actual index
6357 ✗ if (fpindex >= MAX_FRAMEPROP_VARIABLES)
6358 ✗ env->ThrowError("Expr: too many frame property references, maximum reached (%d)", MAX_FRAMEPROP_VARIABLES);
6359 ✗ loadIndex = fpindex++;
6360 ✗ fpnames[key] = loadIndex;
6361 // Register into the list of frame properties to read
6362 // input clip index: srcIndex
6363 // name of frame property: fpname
6364 // variable index to fill: loadindex
6365 ✗ ExprFramePropData fpData;
6366 ✗ fpData.name = fpname;
6367 ✗ fpData.srcIndex = srcIndex;
6368 ✗ fpData.var_index = loadIndex;
6369 ✗ fp.push_back(fpData);
6370 ✗ }
6371 else {
6372 ✗ loadIndex = it->second;
6373 }
6374 ✗ LOAD_OP(opLoadFramePropVar, loadIndex, 0);
6375 ✗ }
6376
7/12
✓ Branch 1505 → 1506 taken 13 times.
✓ Branch 1505 → 1513 taken 4 times.
✓ Branch 1507 → 1508 taken 13 times.
✗ Branch 1507 → 2116 not taken.
✓ Branch 1508 → 1509 taken 13 times.
✗ Branch 1508 → 1512 not taken.
✓ Branch 1510 → 1511 taken 13 times.
✗ Branch 1510 → 2116 not taken.
✗ Branch 1511 → 1512 not taken.
✓ Branch 1511 → 1513 taken 13 times.
✗ Branch 1514 → 1515 not taken.
✓ Branch 1514 → 1550 taken 17 times.
 17 else if (tokenlen >= 2 && (tokens[i][tokenlen - 1] == '^' || tokens[i][tokenlen - 1] == '@'))
6377 {
6378 // storing a variable: A@ .. Z@
6379 // storing a variable and remove from stack: A^..Z^
6380 ✗ auto key = tokens[i].substr(0, tokenlen - 1);
6381 ✗ auto opchar = tokens[i][tokenlen - 1];
6382 ✗ auto it = varnames.find(key);
6383 int loadIndex;
6384 ✗ if (it == varnames.end()) {
6385 ✗ if(!isValidVarName(key))
6386 ✗ env->ThrowError("Expr: invalid variable name '%s'", key.c_str());
6387 // first occurance, insert name and actual index
6388 ✗ if (varindex >= MAX_USER_VARIABLES)
6389 ✗ env->ThrowError("Expr: too many variables, maximum reached (%d)", MAX_USER_VARIABLES);
6390 ✗ loadIndex = varindex++;
6391 ✗ varnames[key] = loadIndex;
6392 }
6393 else {
6394 ✗ loadIndex = it->second;
6395 }
6396 ✗ if (opchar == '^')
6397 ✗ VAR_STORE_SPEC_OP(opStoreVarAndDrop1, loadIndex);
6398 else // if (opchar == '@')
6399 ✗ VAR_STORE_OP(opStoreVar, loadIndex);
6400 ✗ }
6401
1/2
✗ Branch 1552 → 1553 not taken.
✓ Branch 1552 → 1568 taken 17 times.
 17 else if (isValidVarName(tokens[i]))
6402 {
6403 // variable names
6404 // the very end, all reserved expr words were processed
6405 ✗ auto key = tokens[i];
6406 ✗ auto it = varnames.find(key);
6407 ✗ if (it == varnames.end())
6408 ✗ env->ThrowError("Expr: keyword or variable not found: '%s'", key.c_str());
6409 ✗ auto loadIndex = it->second;
6410 ✗ LOAD_OP(opLoadVar, loadIndex, 0);
6411 ✗ }
6412 else {
6413 // parse a number
6414 float f;
6415 17 std::string s;
6416
1/2
✓ Branch 1570 → 1571 taken 17 times.
✗ Branch 1570 → 2095 not taken.
 17 std::istringstream numStream(tokens[i]);
6417
2/4
✓ Branch 1571 → 1572 taken 17 times.
✗ Branch 1571 → 2091 not taken.
✓ Branch 1572 → 1573 taken 17 times.
✗ Branch 1572 → 2091 not taken.
 17 numStream.imbue(std::locale::classic());
6418
3/6
✓ Branch 1574 → 1575 taken 17 times.
✗ Branch 1574 → 2093 not taken.
✓ Branch 1575 → 1576 taken 17 times.
✗ Branch 1575 → 2093 not taken.
✗ Branch 1576 → 1577 not taken.
✓ Branch 1576 → 1580 taken 17 times.
 17 if (!(numStream >> f))
6419 ✗ env->ThrowError("Expr: Failed to convert '%s' to float", tokens[i].c_str());
6420
3/6
✓ Branch 1580 → 1581 taken 17 times.
✗ Branch 1580 → 2093 not taken.
✓ Branch 1581 → 1582 taken 17 times.
✗ Branch 1581 → 2093 not taken.
✗ Branch 1582 → 1583 not taken.
✓ Branch 1582 → 1586 taken 17 times.
 17 if (numStream >> s)
6421 ✗ env->ThrowError("Expr: Failed to convert '%s' to float, not the whole token could be converted", tokens[i].c_str());
6422
1/2
✓ Branch 1587 → 1588 taken 17 times.
✗ Branch 1587 → 2092 not taken.
 17 LOAD_OP(opLoadConst, f, 0);
6423 17 }
6424 }
6425
6426
2/2
✓ Branch 1596 → 1597 taken 33 times.
✓ Branch 1596 → 1742 taken 8 times.
 41 if (tokens.size() > 0) {
6427
1/2
✗ Branch 1597 → 1598 not taken.
✓ Branch 1597 → 1599 taken 33 times.
 33 if (stackSize != 1)
6428 ✗ env->ThrowError("Expr: Stack unbalanced at end of expression (size=%zu). Need to have exactly one value on the stack to return.", stackSize);
6429
6430 // When scale_inputs option was used for scaling input to a common internal range,
6431 // we have to scale pixels before storing them back
6432 // need any conversion?
6433 // or use effectiveTargetBitDepth instead of autoScaleSourceBitDepth
6434 33 const int srcBitDepth = autoScaleSourceBitDepth;
6435 33 const int dstBitDepth = targetBitDepth;
6436 33 const bool use_chroma = chroma; // && !forceNonUV;
6437
6438 33 const bool isfull = autoconv_full_scale;
6439
6440
2/4
✓ Branch 1599 → 1600 taken 33 times.
✗ Branch 1599 → 1601 not taken.
✗ Branch 1600 → 1601 not taken.
✓ Branch 1600 → 1685 taken 33 times.
 33 if (autoconv_conv_int || autoconv_conv_float) {
6441
6442 ✗ if ((targetBitDepth != 32 && autoconv_conv_int) ||
6443 ✗ ((targetBitDepth == 32 && autoconv_conv_float))) {
6444
6445 ✗ bits_conv_constants d;
6446 ✗ get_bits_conv_constants(d, use_chroma, isfull, isfull, srcBitDepth, dstBitDepth);
6447 // chroma is spec: signed. Limited:16-240 is really 128 +/-112. Full:1-255 is really 128+/-127
6448
6449 ✗ if (d.src_offset != 0) {
6450 ✗ LOAD_OP_NOTOKEN(opLoadConst, (float)d.src_offset, 0);
6451 ✗ TWO_ARG_OP_NOTOKEN(opSub);
6452 }
6453 ✗ if (d.mul_factor != 1.0f) {
6454 ✗ LOAD_OP_NOTOKEN(opLoadConst, d.mul_factor, 0);
6455 ✗ TWO_ARG_OP_NOTOKEN(opMul);
6456 }
6457 ✗ if (d.dst_offset != 0) {
6458 ✗ LOAD_OP_NOTOKEN(opLoadConst, (float)d.dst_offset, 0);
6459 ✗ TWO_ARG_OP_NOTOKEN(opAdd);
6460 }
6461 }
6462 }
6463 // "floatUV" -
6464 // this reverses the effect of the 0.5 float-type pre-shift is the pixel load phase
6465 // pre-shifts chroma pixels by +0.5 before applying the expression (see at pixel load),
6466 // then here the result is shifted back by -0.5
6467 // Thus expressions, which rely on a working range of 0..1.0 will work transparently
6468
3/6
✗ Branch 1685 → 1686 not taken.
✓ Branch 1685 → 1687 taken 33 times.
✗ Branch 1686 → 1687 not taken.
✗ Branch 1686 → 1698 not taken.
✓ Branch 1687 → 1688 taken 2 times.
✓ Branch 1687 → 1698 taken 31 times.
 33 if ((!autoconv_conv_float || srcBitDepth == 32) && targetBitDepth == 32) {
6469
1/4
✗ Branch 1688 → 1689 not taken.
✓ Branch 1688 → 1698 taken 2 times.
✗ Branch 1689 → 1690 not taken.
✗ Branch 1689 → 1698 not taken.
 2 if (chroma && shift_float) {
6470 ✗ LOAD_OP_NOTOKEN(opLoadConst, 0.5f, 0);
6471 ✗ TWO_ARG_OP_NOTOKEN(opSub); // at pixel load it was opAdd
6472 }
6473 }
6474
6475
1/4
✗ Branch 1698 → 1699 not taken.
✓ Branch 1698 → 1739 taken 33 times.
✗ Branch 1699 → 1700 not taken.
✗ Branch 1699 → 1739 not taken.
 33 if (clamp_float_i > 0 && targetBitDepth == 32) {
6476 ✗ if (chroma) {
6477 // clamp_float clamp_float_uv -> clamp_float_i clamp range for Y clamp range for UV
6478 // false x 0 0..1 -0.5..+0.5
6479 // true false 1 0..1 -0.5..+0.5
6480 // true true 2 0..1 0..1
6481 ✗ LOAD_OP_NOTOKEN(opLoadConst, clamp_float_i == 2 ? 0.0f : -0.5f, 0);
6482 ✗ TWO_ARG_OP_NOTOKEN(opMax);
6483 ✗ LOAD_OP_NOTOKEN(opLoadConst, clamp_float_i == 2 ? 1.0f : 0.5f, 0);
6484 ✗ TWO_ARG_OP_NOTOKEN(opMin);
6485 }
6486 else
6487 { // luma
6488 ✗ LOAD_OP_NOTOKEN(opLoadConst, 0.0f, 0);
6489 ✗ TWO_ARG_OP_NOTOKEN(opMax);
6490 ✗ LOAD_OP_NOTOKEN(opLoadConst, 1.0f, 0);
6491 ✗ TWO_ARG_OP_NOTOKEN(opMin);
6492 }
6493 }
6494
6495 // and finally store it
6496
1/2
✓ Branch 1740 → 1741 taken 33 times.
✗ Branch 1740 → 2115 not taken.
 33 ops.push_back(storeOp);
6497 }
6498
6499 41 return maxStackSize;
6500 41 }
6501
6502 ✗ static float calculateOneOperand(uint32_t op, float a) {
6503 ✗ switch (op) {
6504 ✗ case opSqrt:
6505 ✗ return std::sqrt(a);
6506 ✗ case opAbs:
6507 ✗ return std::abs(a);
6508 ✗ case opSgn:
6509 ✗ return (a < 0) ? -1.0f : (a > 0) ? 1.0f : 0.0f;
6510 ✗ case opNeg: // Expr "boolean": not
6511 ✗ return (a > 0) ? 0.0f : 1.0f;
6512 ✗ case opNegSign:
6513 ✗ return -a;
6514 ✗ case opExp:
6515 ✗ return std::exp(a);
6516 ✗ case opLog:
6517 ✗ return std::log(a);
6518 ✗ case opSin:
6519 ✗ return std::sin(a);
6520 ✗ case opCos:
6521 ✗ return std::cos(a);
6522 ✗ case opTan:
6523 ✗ return std::tan(a);
6524 ✗ case opAsin:
6525 ✗ return std::asin(a);
6526 ✗ case opAcos:
6527 ✗ return std::acos(a);
6528 ✗ case opAtan:
6529 ✗ return std::atan(a);
6530 ✗ case opRound:
6531 ✗ return std::round(a);
6532 ✗ case opFloor:
6533 ✗ return std::floor(a);
6534 ✗ case opCeil:
6535 ✗ return std::ceil(a);
6536 ✗ case opTrunc:
6537 ✗ return std::trunc(a);
6538 }
6539
6540 ✗ return 0.0f;
6541 }
6542
6543 ✗ static float calculateTwoOperands(uint32_t op, float a, float b) {
6544 ✗ switch (op) {
6545 ✗ case opAdd:
6546 ✗ return a + b;
6547 ✗ case opSub:
6548 ✗ return a - b;
6549 ✗ case opMul:
6550 ✗ return a * b;
6551 ✗ case opDiv:
6552 ✗ return a / b;
6553 ✗ case opFmod:
6554 ✗ return std::fmod(a, b);
6555 ✗ case opMax:
6556 ✗ return std::max(a, b);
6557 ✗ case opMin:
6558 ✗ return std::min(a, b);
6559 ✗ case opGt:
6560 ✗ return (a > b) ? 1.0f : 0.0f;
6561 ✗ case opLt:
6562 ✗ return (a < b) ? 1.0f : 0.0f;
6563 ✗ case opEq:
6564 ✗ return (a == b) ? 1.0f : 0.0f;
6565 ✗ case opNotEq:
6566 ✗ return (a != b) ? 1.0f : 0.0f;
6567 ✗ case opLE:
6568 ✗ return (a <= b) ? 1.0f : 0.0f;
6569 ✗ case opGE:
6570 ✗ return (a >= b) ? 1.0f : 0.0f;
6571 ✗ case opAnd:
6572 ✗ return (a > 0 && b > 0) ? 1.0f : 0.0f;
6573 ✗ case opOr:
6574 ✗ return (a > 0 || b > 0) ? 1.0f : 0.0f;
6575 ✗ case opXor:
6576 ✗ return ((a > 0) != (b > 0)) ? 1.0f : 0.0f;
6577 ✗ case opPow:
6578 ✗ return std::pow(a, b);
6579 ✗ case opAtan2:
6580 ✗ return std::atan2(a, b);
6581 }
6582
6583 ✗ return 0.0f;
6584 }
6585
6586 ✗ static int numOperands(uint32_t op) {
6587 ✗ switch (op) {
6588 ✗ case opLoadConst:
6589 case opLoadSrc8:
6590 case opLoadSrc16:
6591 case opLoadSrcF32:
6592 case opLoadSrcF16:
6593 case opLoadRelSrc8:
6594 case opLoadRelSrc16:
6595 case opLoadRelSrcF32:
6596 case opDup:
6597 case opLoadSpatialX:
6598 case opLoadSpatialY:
6599 case opLoadVar:
6600 case opLoadFramePropVar:
6601 case opLoadInternalVar:
6602 case opSwap:
6603 case opStoreVar:
6604 case opStoreVarAndDrop1:
6605 ✗ return 0;
6606
6607 ✗ case opSqrt:
6608 case opAbs:
6609 case opSgn:
6610 case opNeg:
6611 case opNegSign:
6612 case opExp:
6613 case opLog:
6614 case opSin:
6615 case opCos:
6616 case opTan:
6617 case opAsin:
6618 case opAcos:
6619 case opAtan:
6620 case opRound:
6621 case opFloor:
6622 case opCeil:
6623 case opTrunc:
6624 ✗ return 1;
6625
6626 ✗ case opAdd:
6627 case opSub:
6628 case opMul:
6629 case opDiv:
6630 case opFmod:
6631 case opMax:
6632 case opMin:
6633 case opGt:
6634 case opLt:
6635 case opEq:
6636 case opNotEq:
6637 case opLE:
6638 case opGE:
6639 case opAnd:
6640 case opOr:
6641 case opXor:
6642 case opPow:
6643 case opAtan2:
6644 ✗ return 2;
6645
6646 ✗ case opTernary:
6647 case opClip:
6648 ✗ return 3;
6649 }
6650
6651 ✗ return 0;
6652 }
6653
6654 ✗ static bool isLoadOp(uint32_t op) {
6655 ✗ switch (op) {
6656 ✗ case opLoadConst:
6657 case opLoadSrc8:
6658 case opLoadSrc16:
6659 case opLoadSrcF32:
6660 case opLoadSrcF16:
6661 case opLoadRelSrc8:
6662 case opLoadRelSrc16:
6663 case opLoadRelSrcF32:
6664 case opLoadSpatialX:
6665 case opLoadSpatialY:
6666 case opLoadVar:
6667 case opLoadFramePropVar:
6668 case opLoadInternalVar:
6669 ✗ return true;
6670 }
6671
6672 ✗ return false;
6673 }
6674
6675 ✗ static void findBranches(std::vector<ExprOp>& ops, size_t pos, size_t* start1, size_t* start2, size_t* start3) {
6676 ✗ int operands = numOperands(ops[pos].op);
6677
6678 size_t temp1, temp2, temp3;
6679
6680 ✗ if (operands == 0) {
6681 // dup loadsrc loadrel loadconst swap, storevar
6682 ✗ if (ops[pos].op == opSwap || ops[pos].op == opStoreVar || ops[pos].op == opStoreVarAndDrop1)
6683 {
6684 ✗ findBranches(ops, pos - 1, &temp1, &temp2, &temp3);
6685 ✗ *start1 = temp1;
6686 ✗ if (ops[pos].op == opStoreVarAndDrop1) {
6687 // StoreAndPopVar: opStoreVar + Ignore topmost stack.
6688 // Branch was calculated only for storing its result into var.
6689 // Leaves no trace on stack.
6690 // So we go on with next branch
6691 ✗ pos = *start1;
6692 ✗ findBranches(ops, pos - 1, &temp1, &temp2, &temp3);
6693 ✗ *start1 = temp1;
6694 }
6695 }
6696 else
6697 ✗ *start1 = pos;
6698 }
6699 ✗ else if (operands == 1) {
6700 ✗ if (isLoadOp(ops[pos - 1].op)) {
6701 ✗ *start1 = pos - 1;
6702 }
6703 else {
6704 ✗ findBranches(ops, pos - 1, &temp1, &temp2, &temp3);
6705 ✗ *start1 = temp1;
6706 }
6707 }
6708 ✗ else if (operands == 2) {
6709 ✗ if (isLoadOp(ops[pos - 1].op)) {
6710 ✗ *start2 = pos - 1;
6711 }
6712 else {
6713 ✗ findBranches(ops, pos - 1, &temp1, &temp2, &temp3);
6714 ✗ *start2 = temp1;
6715 }
6716
6717 ✗ if (isLoadOp(ops[*start2 - 1].op)) {
6718 ✗ *start1 = *start2 - 1;
6719 }
6720 else {
6721 ✗ findBranches(ops, *start2 - 1, &temp1, &temp2, &temp3);
6722 ✗ *start1 = temp1;
6723 }
6724 }
6725 ✗ else if (operands == 3) {
6726 ✗ if (isLoadOp(ops[pos - 1].op)) {
6727 ✗ *start3 = pos - 1;
6728 }
6729 else {
6730 ✗ findBranches(ops, pos - 1, &temp1, &temp2, &temp3);
6731 ✗ *start3 = temp1;
6732 }
6733
6734 ✗ if (isLoadOp(ops[*start3 - 1].op)) {
6735 ✗ *start2 = *start3 - 1;
6736 }
6737 else {
6738 ✗ findBranches(ops, *start3 - 1, &temp1, &temp2, &temp3);
6739 ✗ *start2 = temp1;
6740 }
6741
6742 ✗ if (isLoadOp(ops[*start2 - 1].op)) {
6743 ✗ *start1 = *start2 - 1;
6744 }
6745 else {
6746 ✗ findBranches(ops, *start2 - 1, &temp1, &temp2, &temp3);
6747 ✗ *start1 = temp1;
6748 }
6749 }
6750 ✗ }
6751
6752
6753 41 static void foldConstants(std::vector<ExprOp> &ops) {
6754
2/2
✓ Branch 363 → 3 taken 134 times.
✓ Branch 363 → 364 taken 41 times.
 175 for (size_t i = 0; i < ops.size(); i++) {
6755
2/4
✗ Branch 4 → 5 not taken.
✗ Branch 4 → 125 not taken.
✓ Branch 4 → 173 taken 33 times.
✓ Branch 4 → 204 taken 101 times.
 134 switch (ops[i].op) {
6756 // optimize pow
6757 ✗ case opPow:
6758 ✗ if (ops[i - 1].op == opLoadConst) {
6759 ✗ if (ops[i - 1].e.fval == 0.5f) {
6760 // replace pow 0.5 with sqrt
6761 ✗ ops[i].op = opSqrt;
6762 ✗ ops.erase(ops.begin() + i - 1);
6763 ✗ i--;
6764 }
6765 ✗ else if (ops[i - 1].e.fval == 1.0f) {
6766 // replace pow 1 with nothing
6767 ✗ ops.erase(ops.begin() + i - 1, ops.begin() + i + 1);
6768 ✗ i -= 2;
6769 }
6770 ✗ else if (ops[i - 1].e.fval == 2.0f) {
6771 // replace pow 2 with dup *
6772 ✗ ops[i].op = opMul;
6773 ✗ ops[i - 1].op = opDup; ops[i - 1].e.ival = 0; // dup 0
6774 ✗ i--;
6775 }
6776 ✗ else if (ops[i - 1].e.fval == 3.0f) {
6777 // replace pow 3 with dup dup * *
6778 ✗ ops[i].op = opMul;
6779 ✗ ops[i - 1].op = opMul;
6780 ✗ ExprOp extraDup(opDup, 0);
6781 ✗ ops.insert(ops.begin() + i - 1, extraDup);
6782 ✗ ops.insert(ops.begin() + i - 1, extraDup);
6783 ✗ i--;
6784 }
6785 ✗ else if (ops[i - 1].e.fval == 4.0f) {
6786 // replace pow 4 with dup * dup *
6787 ✗ ops[i].op = opMul;
6788 ✗ ops[i - 1].op = opDup; ops[i - 1].e.ival = 0; // dup 0
6789 ✗ ExprOp extraMul(opMul);
6790 ✗ ExprOp extraDup(opDup, 0);
6791 ✗ ops.insert(ops.begin() + i - 1, extraMul);
6792 ✗ ops.insert(ops.begin() + i - 1, extraDup);
6793 ✗ i--;
6794 }
6795 }
6796 ✗ break;
6797 // optimize Mul 1 Div 1, Mul -1, Div -1
6798 ✗ case opMul: case opDiv:
6799 ✗ if (ops[i - 1].op == opLoadConst) {
6800 ✗ if (ops[i - 1].e.fval == 1.0f) {
6801 // replace mul 1 or div 1 with nothing
6802 ✗ ops.erase(ops.begin() + i - 1, ops.begin() + i + 1);
6803 ✗ i -= 2;
6804 }
6805 ✗ else if (ops[i - 1].e.fval == -1.0f) {
6806 // replace mul -1 or div -1 with neg
6807 ✗ ops[i].op = opNegSign;
6808 ✗ ops.erase(ops.begin() + i - 1);
6809 ✗ i--;
6810 }
6811 }
6812 ✗ break;
6813 // optimize Add 0 or Sub 0
6814 33 case opAdd: case opSub:
6815
2/2
✓ Branch 174 → 175 taken 16 times.
✓ Branch 174 → 203 taken 17 times.
 33 if (ops[i - 1].op == opLoadConst) {
6816
1/2
✗ Branch 176 → 177 not taken.
✓ Branch 176 → 203 taken 16 times.
 16 if (ops[i - 1].e.fval == 0.0f) {
6817 // replace add 0 or sub 0 with nothing
6818 ✗ ops.erase(ops.begin() + i - 1, ops.begin() + i + 1);
6819 ✗ i -= 2;
6820 }
6821 }
6822 33 break;
6823
6824 }
6825
6826 // fold constant
6827
3/6
✗ Branch 205 → 206 not taken.
✓ Branch 205 → 217 taken 2 times.
✗ Branch 205 → 239 not taken.
✓ Branch 205 → 263 taken 33 times.
✗ Branch 205 → 299 not taken.
✓ Branch 205 → 361 taken 99 times.
 134 switch (ops[i].op) {
6828 ✗ case opDup:
6829 ✗ if (ops[i - 1].op == opLoadConst && ops[i].e.ival == 0) {
6830 ✗ ops[i] = ops[i - 1];
6831 }
6832 ✗ break;
6833
6834 2 case opSqrt:
6835 case opAbs:
6836 case opSgn:
6837 case opNeg:
6838 case opNegSign:
6839 case opExp:
6840 case opLog:
6841 case opSin:
6842 case opCos:
6843 case opTan:
6844 case opAsin:
6845 case opAcos:
6846 case opAtan:
6847 case opRound:
6848 case opFloor:
6849 case opCeil:
6850 case opTrunc:
6851
1/2
✗ Branch 218 → 219 not taken.
✓ Branch 218 → 238 taken 2 times.
 2 if (ops[i - 1].op == opLoadConst) {
6852 ✗ ops[i].e.fval = calculateOneOperand(ops[i].op, ops[i - 1].e.fval);
6853 ✗ ops[i].op = opLoadConst;
6854 ✗ ops.erase(ops.begin() + i - 1);
6855 ✗ i--;
6856 }
6857 2 break;
6858
6859 ✗ case opSwap:
6860 ✗ if (ops[i - 2].op == opLoadConst && ops[i - 1].op == opLoadConst && ops[i].e.ival == 1) {
6861 ✗ const float temp = ops[i - 2].e.fval;
6862 ✗ ops[i - 2].e.fval = ops[i - 1].e.fval;
6863 ✗ ops[i - 1].e.fval = temp;
6864 ✗ ops.erase(ops.begin() + i);
6865 ✗ i--;
6866 }
6867 ✗ break;
6868
6869 33 case opAdd:
6870 case opSub:
6871 case opMul:
6872 case opDiv:
6873 case opFmod:
6874 case opMax:
6875 case opMin:
6876 case opGt:
6877 case opLt:
6878 case opEq:
6879 case opNotEq:
6880 case opLE:
6881 case opGE:
6882 case opAnd:
6883 case opOr:
6884 case opXor:
6885 case opPow:
6886 case opAtan2:
6887
4/6
✓ Branch 264 → 265 taken 1 time.
✓ Branch 264 → 268 taken 32 times.
✗ Branch 266 → 267 not taken.
✓ Branch 266 → 268 taken 1 time.
✗ Branch 269 → 270 not taken.
✓ Branch 269 → 298 taken 33 times.
 33 if (ops[i - 2].op == opLoadConst && ops[i - 1].op == opLoadConst) {
6888 ✗ ops[i].e.fval = calculateTwoOperands(ops[i].op, ops[i - 2].e.fval, ops[i - 1].e.fval);
6889 ✗ ops[i].op = opLoadConst;
6890 ✗ ops.erase(ops.begin() + i - 2, ops.begin() + i);
6891 ✗ i -= 2;
6892 }
6893 33 break;
6894
6895 ✗ case opTernary:
6896 size_t start1, start2, start3;
6897 ✗ findBranches(ops, i, &start1, &start2, &start3);
6898 // start1/2/3: condition/true/false branch
6899 ✗ if (ops[start2 - 1].op == opLoadConst) { // condition expression is a single constant
6900 ✗ ops.erase(ops.begin() + i); // erase ternary op
6901 ✗ if (ops[start1].e.fval > 0.0f) { // condition is constant 'true'
6902 // start1 is start2 - 1
6903 ✗ ops.erase(ops.begin() + start3, ops.begin() + i); // erase 'false' branch
6904 ✗ i = start3;
6905 } else {
6906 ✗ ops.erase(ops.begin() + start2, ops.begin() + start3); // erase 'true' branch
6907 ✗ i -= start3 - start2;
6908 }
6909 ✗ ops.erase(ops.begin() + start1); // erase constant
6910 ✗ i -= 2;
6911 }
6912 ✗ break;
6913 }
6914 }
6915 41 }
6916
6917 15 Exprfilter::Exprfilter(const std::vector<PClip>& _child_array, const std::vector<std::string>& _expr_array, const char *_newformat, const bool _optAvx2,
6918 15 const bool _optSingleMode, const bool _optSSE2, const bool _optVectorC, const std::string _scale_inputs, const int _clamp_float_i, const int _lutmode, IScriptEnvironment *env) :
6919
1/2
✓ Branch 4 → 5 taken 15 times.
✗ Branch 4 → 457 not taken.
 15 children(_child_array), expressions(_expr_array), optAvx2(_optAvx2), optSingleMode(_optSingleMode), optSSE2(_optSSE2),
6920
3/6
✓ Branch 3 → 4 taken 15 times.
✗ Branch 3 → 459 not taken.
✓ Branch 5 → 6 taken 15 times.
✗ Branch 5 → 455 not taken.
✓ Branch 6 → 7 taken 15 times.
✗ Branch 6 → 453 not taken.
 30 optVectorC(_optVectorC), scale_inputs(_scale_inputs), clamp_float_i(_clamp_float_i), lutmode(_lutmode) {
6921
6922
1/2
✓ Branch 9 → 10 taken 15 times.
✗ Branch 9 → 451 not taken.
 15 vi = children[0]->GetVideoInfo();
6923 15 d.vi = vi;
6924
2/4
✓ Branch 10 → 11 taken 15 times.
✗ Branch 10 → 12 not taken.
✗ Branch 11 → 12 not taken.
✓ Branch 11 → 13 taken 15 times.
 15 if (lutmode < 0 || lutmode>2)
6925 ✗ env->ThrowError("'Expr: 'lut' can be 0 (no lut), 1 (lut_x) or 2 (lut_xy)");
6926
5/8
✓ Branch 13 → 14 taken 1 time.
✓ Branch 13 → 17 taken 14 times.
✓ Branch 14 → 15 taken 1 time.
✗ Branch 14 → 451 not taken.
✗ Branch 15 → 16 not taken.
✓ Branch 15 → 17 taken 1 time.
✗ Branch 18 → 19 not taken.
✓ Branch 18 → 20 taken 15 times.
 15 if (lutmode == 1 && vi.BitsPerComponent() == 32)
6927 ✗ lutmode = 0; // fallback to realtime
6928
2/8
✗ Branch 20 → 21 not taken.
✓ Branch 20 → 24 taken 15 times.
✗ Branch 21 → 22 not taken.
✗ Branch 21 → 451 not taken.
✗ Branch 22 → 23 not taken.
✗ Branch 22 → 24 not taken.
✗ Branch 25 → 26 not taken.
✓ Branch 25 → 27 taken 15 times.
 15 if (lutmode == 2 && vi.BitsPerComponent() > 14)
6929 ✗ lutmode = 0; // fallback to realtime
6930 15 d.lutmode = lutmode;
6931
6932
2/2
✓ Branch 29 → 28 taken 60 times.
✓ Branch 29 → 30 taken 15 times.
 75 for (int i = 0; i < 4; i++)
6933 60 d.luts[i] = nullptr;
6934
6935 // parse "scale_inputs"
6936 15 autoconv_full_scale = false;
6937 15 autoconv_conv_float = false;
6938 15 autoconv_conv_int = false;
6939 15 shift_float = false;
6940
6941
2/4
✓ Branch 30 → 31 taken 15 times.
✗ Branch 30 → 451 not taken.
✗ Branch 31 → 32 not taken.
✓ Branch 31 → 33 taken 15 times.
 15 if (scale_inputs == "allf") {
6942 ✗ autoconv_full_scale = true;
6943 ✗ autoconv_conv_int = true;
6944 ✗ autoconv_conv_float = true;
6945 }
6946
2/4
✓ Branch 33 → 34 taken 15 times.
✗ Branch 33 → 451 not taken.
✗ Branch 34 → 35 not taken.
✓ Branch 34 → 36 taken 15 times.
 15 else if (scale_inputs == "intf") {
6947 ✗ autoconv_full_scale = true;
6948 ✗ autoconv_conv_int = true;
6949 }
6950
2/4
✓ Branch 36 → 37 taken 15 times.
✗ Branch 36 → 451 not taken.
✗ Branch 37 → 38 not taken.
✓ Branch 37 → 39 taken 15 times.
 15 else if (scale_inputs == "floatf") {
6951 ✗ autoconv_full_scale = true;
6952 ✗ autoconv_conv_float = true;
6953 }
6954
2/4
✓ Branch 39 → 40 taken 15 times.
✗ Branch 39 → 451 not taken.
✗ Branch 40 → 41 not taken.
✓ Branch 40 → 42 taken 15 times.
 15 else if (scale_inputs == "all") {
6955 ✗ autoconv_conv_int = true;
6956 ✗ autoconv_conv_float = true;
6957 }
6958
2/4
✓ Branch 42 → 43 taken 15 times.
✗ Branch 42 → 451 not taken.
✗ Branch 43 → 44 not taken.
✓ Branch 43 → 45 taken 15 times.
 15 else if (scale_inputs == "int") {
6959 ✗ autoconv_conv_int = true;
6960 }
6961
2/4
✓ Branch 45 → 46 taken 15 times.
✗ Branch 45 → 451 not taken.
✗ Branch 46 → 47 not taken.
✓ Branch 46 → 48 taken 15 times.
 15 else if (scale_inputs == "float") {
6962 ✗ autoconv_conv_float = true;
6963 }
6964
2/4
✓ Branch 48 → 49 taken 15 times.
✗ Branch 48 → 451 not taken.
✗ Branch 49 → 50 not taken.
✓ Branch 49 → 51 taken 15 times.
 15 else if (scale_inputs == "floatuv") {
6965 ✗ autoconv_conv_float = false; // !! really
6966 // like in masktools2 2.2.20+
6967 ✗ shift_float = true; // !!
6968 }
6969
2/4
✓ Branch 51 → 52 taken 15 times.
✗ Branch 51 → 451 not taken.
✗ Branch 52 → 53 not taken.
✓ Branch 52 → 54 taken 15 times.
 15 else if (scale_inputs != "none") {
6970 ✗ env->ThrowError("Expr: scale_inputs must be 'all','allf','int','intf','float','floatf','floatUV' or 'none'");
6971 }
6972
6973 try {
6974 15 d.numInputs = (int)children.size(); // d->numInputs = vsapi->propNumElements(in, "clips");
6975
1/2
✗ Branch 55 → 56 not taken.
✓ Branch 55 → 57 taken 15 times.
 15 if (d.numInputs > 26)
6976 ✗ env->ThrowError("Expr: More than 26 input clips provided");
6977
6978
2/2
✓ Branch 61 → 58 taken 15 times.
✓ Branch 61 → 62 taken 15 times.
 30 for (int i = 0; i < d.numInputs; i++)
6979
1/2
✓ Branch 59 → 60 taken 15 times.
✗ Branch 59 → 424 not taken.
 15 d.clips[i] = children[i];
6980
6981
2/2
✓ Branch 62 → 63 taken 1 time.
✓ Branch 62 → 65 taken 14 times.
 15 if (d.lutmode > 0) {
6982
1/2
✗ Branch 63 → 64 not taken.
✓ Branch 63 → 65 taken 1 time.
 1 if(d.numInputs != d.lutmode)
6983 ✗ env->ThrowError("Expr lut: number of input clips must be the same as LUT's dimension. LUT is %dD. Passed clip(s): %d", d.lutmode, d.numInputs);
6984 }
6985
6986 // checking formats
6987 15 const VideoInfo* vi_array[MAX_EXPR_INPUTS] = {};
6988
2/2
✓ Branch 72 → 66 taken 15 times.
✓ Branch 72 → 73 taken 15 times.
 30 for (int i = 0; i < d.numInputs; i++)
6989
1/2
✓ Branch 67 → 68 taken 15 times.
✗ Branch 67 → 71 not taken.
 15 if (d.clips[i])
6990
1/2
✓ Branch 69 → 70 taken 15 times.
✗ Branch 69 → 424 not taken.
 15 vi_array[i] = &d.clips[i]->GetVideoInfo();
6991
6992
6993 15 int planes_y[4] = { PLANAR_Y, PLANAR_U, PLANAR_V, PLANAR_A };
6994 15 int planes_r[4] = { PLANAR_G, PLANAR_B, PLANAR_R, PLANAR_A }; // for checking GBR order is OK
6995
5/8
✓ Branch 73 → 74 taken 15 times.
✗ Branch 73 → 424 not taken.
✓ Branch 74 → 75 taken 6 times.
✓ Branch 74 → 77 taken 9 times.
✓ Branch 75 → 76 taken 6 times.
✗ Branch 75 → 424 not taken.
✗ Branch 76 → 77 not taken.
✓ Branch 76 → 78 taken 6 times.
 15 int *plane_enums = (d.vi.IsYUV() || d.vi.IsYUVA()) ? planes_y : planes_r;
6996 15 const int plane_enum = plane_enums[1]; // for subsampling check U only
6997
6998 // check all clips against first one
6999
2/2
✓ Branch 115 → 80 taken 15 times.
✓ Branch 115 → 116 taken 15 times.
 30 for (int i = 0; i < d.numInputs; i++) {
7000
7001
5/8
✓ Branch 80 → 81 taken 15 times.
✗ Branch 80 → 424 not taken.
✓ Branch 81 → 82 taken 6 times.
✓ Branch 81 → 84 taken 9 times.
✓ Branch 82 → 83 taken 6 times.
✗ Branch 82 → 424 not taken.
✗ Branch 83 → 84 not taken.
✓ Branch 83 → 85 taken 6 times.
 15 int *plane_enums_i = (vi_array[i]->IsYUV() || vi_array[i]->IsYUVA()) ? planes_y : planes_r;
7002 15 const int plane_enum_i = plane_enums_i[1];
7003
7004
2/4
✓ Branch 86 → 87 taken 15 times.
✗ Branch 86 → 424 not taken.
✓ Branch 87 → 88 taken 15 times.
✗ Branch 87 → 424 not taken.
 15 if (vi_array[0]->NumComponents() != vi_array[i]->NumComponents() // number of planes should match
7005 15 ||
7006
3/4
✓ Branch 89 → 90 taken 15 times.
✗ Branch 89 → 424 not taken.
✓ Branch 90 → 91 taken 10 times.
✓ Branch 90 → 97 taken 5 times.
 15 ( !vi_array[0]->IsY() && ( // no subsampling for Y
7007
3/6
✓ Branch 91 → 92 taken 10 times.
✗ Branch 91 → 424 not taken.
✓ Branch 92 → 93 taken 10 times.
✗ Branch 92 → 424 not taken.
✓ Branch 93 → 94 taken 10 times.
✗ Branch 93 → 99 not taken.
 10 vi_array[0]->GetPlaneWidthSubsampling(plane_enum) != vi_array[i]->GetPlaneWidthSubsampling(plane_enum_i)
7008
3/6
✓ Branch 94 → 95 taken 10 times.
✗ Branch 94 → 424 not taken.
✓ Branch 95 → 96 taken 10 times.
✗ Branch 95 → 424 not taken.
✓ Branch 96 → 97 taken 10 times.
✗ Branch 96 → 99 not taken.
 10 || vi_array[0]->GetPlaneHeightSubsampling(plane_enum) != vi_array[i]->GetPlaneHeightSubsampling(plane_enum_i)
7009 )
7010 )
7011
1/2
✓ Branch 97 → 98 taken 15 times.
✗ Branch 97 → 99 not taken.
 15 || vi_array[0]->width != vi_array[i]->width
7012
3/6
✓ Branch 88 → 89 taken 15 times.
✗ Branch 88 → 99 not taken.
✗ Branch 98 → 99 not taken.
✓ Branch 98 → 100 taken 15 times.
✗ Branch 101 → 102 not taken.
✓ Branch 101 → 103 taken 15 times.
 30 || vi_array[0]->height != vi_array[i]->height)
7013 ✗ env->ThrowError("Expr: All inputs must have the same number of planes and the same dimensions, subsampling included");
7014
7015
6/10
✓ Branch 103 → 104 taken 15 times.
✗ Branch 103 → 424 not taken.
✓ Branch 104 → 105 taken 6 times.
✓ Branch 104 → 108 taken 9 times.
✓ Branch 105 → 106 taken 6 times.
✗ Branch 105 → 424 not taken.
✗ Branch 106 → 107 not taken.
✓ Branch 106 → 108 taken 6 times.
✗ Branch 109 → 110 not taken.
✓ Branch 109 → 111 taken 15 times.
 15 if (vi_array[i]->IsRGB() && !vi_array[i]->IsPlanar())
7016 ✗ env->ThrowError("Expr: No packed RGB format allowed for clip #%d, use planar RGB instead",i+1);
7017
2/4
✓ Branch 111 → 112 taken 15 times.
✗ Branch 111 → 424 not taken.
✗ Branch 112 → 113 not taken.
✓ Branch 112 → 114 taken 15 times.
 15 if (vi_array[i]->IsYUY2())
7018 ✗ env->ThrowError("Expr: YUY2 format not allowed for clip #%d", i+1);
7019 }
7020
7021 // format override
7022
2/2
✓ Branch 116 → 117 taken 2 times.
✓ Branch 116 → 172 taken 13 times.
 15 if (_newformat != nullptr) {
7023
1/2
✓ Branch 117 → 118 taken 2 times.
✗ Branch 117 → 424 not taken.
 2 int pixel_type = GetPixelTypeFromName(_newformat);
7024
1/2
✗ Branch 118 → 119 not taken.
✓ Branch 118 → 120 taken 2 times.
 2 if (pixel_type == VideoInfo::CS_UNKNOWN)
7025 ✗ env->ThrowError("Expr: Invalid video format string parameter");
7026 2 d.vi.pixel_type = pixel_type;
7027
3/10
✓ Branch 120 → 121 taken 2 times.
✗ Branch 120 → 424 not taken.
✗ Branch 121 → 122 not taken.
✓ Branch 121 → 125 taken 2 times.
✗ Branch 122 → 123 not taken.
✗ Branch 122 → 424 not taken.
✗ Branch 123 → 124 not taken.
✗ Branch 123 → 125 not taken.
✗ Branch 126 → 127 not taken.
✓ Branch 126 → 128 taken 2 times.
 2 if(d.vi.IsRGB() && !d.vi.IsPlanar())
7028 ✗ env->ThrowError("Expr: No packed RGB format allowed");
7029
2/4
✓ Branch 128 → 129 taken 2 times.
✗ Branch 128 → 424 not taken.
✗ Branch 129 → 130 not taken.
✓ Branch 129 → 131 taken 2 times.
 2 if (d.vi.IsYUY2())
7030 ✗ env->ThrowError("Expr: YUY2 format not allowed");
7031
7032
1/2
✓ Branch 131 → 132 taken 2 times.
✗ Branch 131 → 424 not taken.
 2 const bool isSinglePlaneInput = vi_array[0]->IsY();
7033
7034 // input number of planes >= output planes
7035
5/10
✓ Branch 132 → 133 taken 2 times.
✗ Branch 132 → 137 not taken.
✓ Branch 133 → 134 taken 2 times.
✗ Branch 133 → 424 not taken.
✓ Branch 134 → 135 taken 2 times.
✗ Branch 134 → 424 not taken.
✗ Branch 135 → 136 not taken.
✓ Branch 135 → 137 taken 2 times.
✗ Branch 138 → 139 not taken.
✓ Branch 138 → 140 taken 2 times.
 2 if (!isSinglePlaneInput && vi_array[0]->NumComponents() < d.vi.NumComponents())
7036 ✗ env->ThrowError("Expr: number of planes in input should be greater than or equal than of output");
7037
7038 // subsampling should match
7039
4/8
✓ Branch 140 → 141 taken 2 times.
✗ Branch 140 → 424 not taken.
✓ Branch 141 → 142 taken 2 times.
✗ Branch 141 → 144 not taken.
✓ Branch 142 → 143 taken 2 times.
✗ Branch 142 → 424 not taken.
✗ Branch 143 → 144 not taken.
✓ Branch 143 → 145 taken 2 times.
 2 int *plane_enums_s = (vi_array[0]->IsYUV() || vi_array[0]->IsYUVA()) ? planes_y : planes_r;
7040
4/8
✓ Branch 146 → 147 taken 2 times.
✗ Branch 146 → 424 not taken.
✓ Branch 147 → 148 taken 2 times.
✗ Branch 147 → 150 not taken.
✓ Branch 148 → 149 taken 2 times.
✗ Branch 148 → 424 not taken.
✓ Branch 149 → 150 taken 2 times.
✗ Branch 149 → 151 not taken.
 2 int *plane_enums_d = (d.vi.IsYUV() || d.vi.IsYUVA()) ? planes_y : planes_r;
7041
3/4
✓ Branch 169 → 170 taken 10 times.
✗ Branch 169 → 424 not taken.
✓ Branch 170 → 153 taken 8 times.
✓ Branch 170 → 171 taken 2 times.
 10 for (int p = 0; p < d.vi.NumComponents(); p++) {
7042
1/2
✗ Branch 153 → 154 not taken.
✓ Branch 153 → 155 taken 8 times.
 8 const int plane_enum_s = isSinglePlaneInput ? plane_enums_s[0] : plane_enums_s[p]; // for Y inputs, reference is Y for each output plane
7043 8 const int plane_enum_d = plane_enums_d[p];
7044
2/4
✓ Branch 156 → 157 taken 8 times.
✗ Branch 156 → 424 not taken.
✓ Branch 157 → 158 taken 8 times.
✗ Branch 157 → 424 not taken.
 8 if (vi_array[0]->GetPlaneWidthSubsampling(plane_enum_s) != d.vi.GetPlaneWidthSubsampling(plane_enum_d)
7045
5/10
✓ Branch 158 → 159 taken 8 times.
✗ Branch 158 → 162 not taken.
✓ Branch 159 → 160 taken 8 times.
✗ Branch 159 → 424 not taken.
✓ Branch 160 → 161 taken 8 times.
✗ Branch 160 → 424 not taken.
✗ Branch 161 → 162 not taken.
✓ Branch 161 → 163 taken 8 times.
✗ Branch 164 → 165 not taken.
✓ Branch 164 → 168 taken 8 times.
 8 || vi_array[0]->GetPlaneHeightSubsampling(plane_enum_s) != d.vi.GetPlaneHeightSubsampling(plane_enum_d)) {
7046 ✗ if(isSinglePlaneInput)
7047 ✗ env->ThrowError("Expr: output must not be a subsampled format for Y-only input(s)");
7048 else
7049 ✗ env->ThrowError("Expr: inputs and output must have the same subsampling");
7050 }
7051 }
7052
7053 2 vi = d.vi;
7054 }
7055
7056 // check expression count, duplicate omitted expressions from previous one
7057 15 int nexpr = (int)expressions.size();
7058
2/4
✓ Branch 173 → 174 taken 15 times.
✗ Branch 173 → 424 not taken.
✗ Branch 174 → 175 not taken.
✓ Branch 174 → 176 taken 15 times.
 15 if (nexpr > d.vi.NumComponents()) // ->numPlanes)
7059 ✗ env->ThrowError("Expr: More expressions given than there are planes");
7060
7061
2/2
✓ Branch 179 → 177 taken 60 times.
✓ Branch 179 → 180 taken 15 times.
 150 std::string expr[4]; // 4th: alpha
7062
2/2
✓ Branch 184 → 181 taken 37 times.
✓ Branch 184 → 185 taken 15 times.
 52 for (int i = 0; i < nexpr; i++)
7063
1/2
✓ Branch 182 → 183 taken 37 times.
✗ Branch 182 → 417 not taken.
 37 expr[i] = expressions[i];
7064
2/2
✓ Branch 185 → 186 taken 5 times.
✓ Branch 185 → 188 taken 10 times.
 15 if (nexpr == 1) {
7065
1/2
✓ Branch 186 → 187 taken 5 times.
✗ Branch 186 → 417 not taken.
 5 expr[1] = expr[0];
7066
1/2
✓ Branch 187 → 190 taken 5 times.
✗ Branch 187 → 417 not taken.
 5 expr[2] = expr[0]; // e.g. exprU = exprV = exprY
7067 }
7068
1/2
✗ Branch 188 → 189 not taken.
✓ Branch 188 → 190 taken 10 times.
 10 else if (nexpr == 2) {
7069 ✗ expr[2] = expr[1]; // e.g. exprV = exprU
7070 }
7071
2/2
✓ Branch 190 → 191 taken 13 times.
✓ Branch 190 → 192 taken 2 times.
 15 if(nexpr <= 3)
7072
1/2
✓ Branch 191 → 192 taken 13 times.
✗ Branch 191 → 417 not taken.
 13 expr[3] = ""; // do not use previous expression to alpha expr. Default: "" (copy)
7073
7074 // default: all clips unused
7075
2/2
✓ Branch 194 → 193 taken 390 times.
✓ Branch 194 → 195 taken 15 times.
 405 for (int i = 0; i < MAX_EXPR_INPUTS; i++) {
7076 390 d.clipsUsed[i] = false;
7077 }
7078
7079
3/4
✓ Branch 205 → 206 taken 56 times.
✗ Branch 205 → 417 not taken.
✓ Branch 206 → 196 taken 41 times.
✓ Branch 206 → 207 taken 15 times.
 56 for (int i = 0; i < d.vi.NumComponents(); i++) {
7080
2/2
✓ Branch 197 → 198 taken 33 times.
✓ Branch 197 → 199 taken 8 times.
 41 if (!expr[i].empty()) {
7081 33 d.plane[i] = poProcess;
7082 }
7083 else {
7084
3/6
✓ Branch 199 → 200 taken 8 times.
✗ Branch 199 → 417 not taken.
✓ Branch 200 → 201 taken 8 times.
✗ Branch 200 → 417 not taken.
✓ Branch 201 → 202 taken 8 times.
✗ Branch 201 → 203 not taken.
 8 if (d.vi.BitsPerComponent() == vi_array[0]->BitsPerComponent()) {
7085 8 d.plane[i] = poCopy; // copy only when target clip format bit depth == 1st clip's bit depth
7086 8 d.planeCopySourceClip[i] = 0; // default source clip from empty expression: first one
7087 8 d.clipsUsed[0] = true; // mark clip to have its GetFrame
7088 }
7089 else
7090 ✗ d.plane[i] = poUndefined;
7091 }
7092 }
7093
7094
6/8
✓ Branch 207 → 208 taken 15 times.
✗ Branch 207 → 417 not taken.
✓ Branch 208 → 209 taken 6 times.
✓ Branch 208 → 211 taken 9 times.
✓ Branch 209 → 210 taken 6 times.
✗ Branch 209 → 417 not taken.
✓ Branch 210 → 211 taken 2 times.
✓ Branch 210 → 212 taken 4 times.
 15 int* plane_enums_d = (d.vi.IsYUV() || d.vi.IsYUVA()) ? planes_y : planes_r;
7095
7096 15 d.maxStackSize = 0;
7097
3/4
✓ Branch 349 → 350 taken 56 times.
✗ Branch 349 → 417 not taken.
✓ Branch 350 → 214 taken 41 times.
✓ Branch 350 → 351 taken 15 times.
 56 for (int i = 0; i < d.vi.NumComponents(); i++) {
7098 41 const int plane_enum_s = plane_enums[i];
7099 41 const int plane_enum = plane_enums_d[i];
7100
1/2
✓ Branch 214 → 215 taken 41 times.
✗ Branch 214 → 417 not taken.
 41 const int planewidth = d.vi.width >> d.vi.GetPlaneWidthSubsampling(plane_enum);
7101
1/2
✓ Branch 215 → 216 taken 41 times.
✗ Branch 215 → 417 not taken.
 41 const int planeheight = d.vi.height >> d.vi.GetPlaneHeightSubsampling(plane_enum);
7102
4/4
✓ Branch 216 → 217 taken 37 times.
✓ Branch 216 → 218 taken 4 times.
✓ Branch 217 → 218 taken 4 times.
✓ Branch 217 → 219 taken 33 times.
 41 const bool chroma = (plane_enum_s == PLANAR_U || plane_enum_s == PLANAR_V);
7103
1/2
✓ Branch 221 → 222 taken 41 times.
✗ Branch 221 → 409 not taken.
 41 d.maxStackSize = std::max(parseExpression(expr[i], d.ops[i], d.frameprops[i], vi_array, &d.vi, getStoreOp(&d.vi), d.numInputs, planewidth, planeheight, chroma,
7104 41 autoconv_full_scale, autoconv_conv_int, autoconv_conv_float, clamp_float_i, shift_float, d.lutmode,
7105
1/2
✓ Branch 220 → 221 taken 41 times.
✗ Branch 220 → 409 not taken.
 41 env), d.maxStackSize);
7106
1/2
✓ Branch 223 → 224 taken 41 times.
✗ Branch 223 → 417 not taken.
 41 foldConstants(d.ops[i]);
7107
7108 // optimize constant store, change operation to "fill"
7109
6/8
✓ Branch 224 → 225 taken 33 times.
✓ Branch 224 → 230 taken 8 times.
✓ Branch 226 → 227 taken 2 times.
✓ Branch 226 → 230 taken 31 times.
✗ Branch 228 → 229 not taken.
✓ Branch 228 → 230 taken 2 times.
✗ Branch 231 → 232 not taken.
✓ Branch 231 → 241 taken 41 times.
 41 if (d.plane[i] == poProcess && d.ops[i].size() == 2 && d.ops[i][0].op == opLoadConst) {
7110 ✗ uint32_t op = d.ops[i][1].op;
7111 ✗ if (op == opStore8 || op == opStore10 || op == opStore12 || op == opStore14 || op == opStore16 || op == opStoreF32)
7112 {
7113 ✗ d.plane[i] = poFill;
7114 ✗ d.planeFillValue[i] = d.ops[i][0].e.fval;
7115 }
7116 }
7117
7118 // optimize single clip letter in expression: Load-Store. Change operation to "copy"
7119 // no relative loads here
7120
2/2
✓ Branch 243 → 244 taken 2 times.
✓ Branch 243 → 267 taken 31 times.
 33 if (d.plane[i] == poProcess && d.ops[i].size() == 2 &&
7121
7/12
✓ Branch 241 → 242 taken 33 times.
✓ Branch 241 → 267 taken 8 times.
✓ Branch 245 → 246 taken 2 times.
✗ Branch 245 → 252 not taken.
✓ Branch 247 → 248 taken 2 times.
✗ Branch 247 → 252 not taken.
✓ Branch 249 → 250 taken 2 times.
✗ Branch 249 → 252 not taken.
✗ Branch 251 → 252 not taken.
✓ Branch 251 → 267 taken 2 times.
✗ Branch 268 → 269 not taken.
✓ Branch 268 → 274 taken 41 times.
 74 (d.ops[i][0].op == opLoadSrc8 || d.ops[i][0].op == opLoadSrc16 || d.ops[i][0].op == opLoadSrcF16 || d.ops[i][0].op == opLoadSrcF32) &&
7122 ✗ (d.ops[i][1].op == opStore8 || d.ops[i][1].op == opStore10 || d.ops[i][1].op == opStore12 || d.ops[i][1].op == opStore14 || d.ops[i][1].op == opStore16 || d.ops[i][1].op == opStoreF16 || d.ops[i][1].op == opStoreF32))
7123 {
7124 ✗ const int sourceClip = d.ops[i][0].e.ival;
7125 // check target vs source bit depth
7126 ✗ if(d.vi.BitsPerComponent() == vi_array[sourceClip]->BitsPerComponent()) // no 16bit float in avs+
7127 {
7128 ✗ d.plane[i] = poCopy;
7129 ✗ d.planeCopySourceClip[i] = sourceClip;
7130 ✗ d.clipsUsed[sourceClip] = true; // mark clip to have its GetFrame
7131 }
7132 }
7133
7134 // optimize: mark referenced input clips in order to not call GetFrame for unused inputs
7135
2/2
✓ Branch 274 → 275 taken 40 times.
✓ Branch 274 → 306 taken 1 time.
 41 if (lutmode == 0) {
7136
2/2
✓ Branch 288 → 276 taken 130 times.
✓ Branch 288 → 289 taken 40 times.
 170 for (size_t j = 0; j < d.ops[i].size(); j++) {
7137 130 const uint32_t op = d.ops[i][j].op;
7138
9/10
✓ Branch 277 → 278 taken 114 times.
✓ Branch 277 → 284 taken 16 times.
✓ Branch 278 → 279 taken 102 times.
✓ Branch 278 → 284 taken 12 times.
✓ Branch 279 → 280 taken 102 times.
✗ Branch 279 → 284 not taken.
✓ Branch 280 → 281 taken 100 times.
✓ Branch 280 → 284 taken 2 times.
✓ Branch 281 → 282 taken 98 times.
✓ Branch 281 → 284 taken 2 times.
 130 if (op == opLoadSrc8 || op == opLoadSrc16 || op == opLoadSrcF16 || op == opLoadSrcF32 ||
7139
2/4
✓ Branch 282 → 283 taken 98 times.
✗ Branch 282 → 284 not taken.
✗ Branch 283 → 284 not taken.
✓ Branch 283 → 286 taken 98 times.
 98 op == opLoadRelSrc8 || op == opLoadRelSrc16 || op == opLoadRelSrcF32)
7140 {
7141 32 const int sourceClip = d.ops[i][j].e.ival;
7142 32 d.clipsUsed[sourceClip] = true;
7143 }
7144 }
7145 // input clips with frame property access are used as well
7146
1/4
✗ Branch 293 → 294 not taken.
✗ Branch 293 → 410 not taken.
✗ Branch 304 → 291 not taken.
✓ Branch 304 → 305 taken 40 times.
 80 for (auto framePropToRead : d.frameprops[i]) {
7147 ✗ const int sourceClip = framePropToRead.srcIndex;
7148 ✗ d.clipsUsed[sourceClip] = true;
7149 ✗ }
7150 }
7151
7152
2/2
✓ Branch 306 → 307 taken 1 time.
✓ Branch 306 → 325 taken 40 times.
 41 if (lutmode > 0) {
7153
2/2
✓ Branch 309 → 308 taken 1 time.
✓ Branch 309 → 310 taken 1 time.
 2 for (int i = 0; i < lutmode; i++) // lut: always get. Needed for the init
7154 1 d.clipsUsed[i] = true;
7155 // * bit depth of input clip(s) and output must match
7156 bool lut_ok =
7157 1 (d.numInputs == 1
7158
3/6
✓ Branch 311 → 312 taken 1 time.
✗ Branch 311 → 417 not taken.
✓ Branch 312 → 313 taken 1 time.
✗ Branch 312 → 417 not taken.
✗ Branch 313 → 314 not taken.
✓ Branch 313 → 321 taken 1 time.
 1 && d.vi.BitsPerComponent() == vi_array[0]->BitsPerComponent())
7159
1/2
✓ Branch 310 → 311 taken 1 time.
✗ Branch 310 → 314 not taken.
 2 ||
7160 (
7161 ✗ d.numInputs == 2
7162 ✗ && d.vi.BitsPerComponent() == vi_array[0]->BitsPerComponent()
7163 ✗ && d.vi.BitsPerComponent() == vi_array[1]->BitsPerComponent()
7164 1 );
7165
1/2
✗ Branch 323 → 324 not taken.
✓ Branch 323 → 325 taken 1 time.
 1 if(!lut_ok)
7166 ✗ env->ThrowError("Expr: error in lut mode: input bit depths and output bit depth must be the same");
7167 }
7168
7169
7170 #ifdef INTEL_INTRINSICS
7171 // Check CPU instuction level constraints:
7172 // opLoadRel8/16/32: minimum SSSE3 (pshufb, alignr) for SIMD, and no AVX2 support
7173 // round, floor, ceil, trunc: minimun SSE4.1
7174 // Trig.func: C only
7175 41 d.planeOptAvx2[i] = optAvx2;
7176 41 d.planeOptSSE2[i] = optSSE2;
7177
2/2
✓ Branch 347 → 326 taken 134 times.
✓ Branch 347 → 348 taken 41 times.
 175 for (size_t j = 0; j < d.ops[i].size(); j++) {
7178 134 const uint32_t op = d.ops[i][j].op;
7179
4/6
✓ Branch 327 → 328 taken 132 times.
✓ Branch 327 → 330 taken 2 times.
✓ Branch 328 → 329 taken 132 times.
✗ Branch 328 → 330 not taken.
✗ Branch 329 → 330 not taken.
✓ Branch 329 → 333 taken 132 times.
 134 if (op == opLoadRelSrc8 || op == opLoadRelSrc16 || op == opLoadRelSrcF32)
7180 {
7181 2 d.planeOptAvx2[i] = false; // avx2 path not implemented
7182
2/4
✓ Branch 330 → 331 taken 2 times.
✗ Branch 330 → 417 not taken.
✗ Branch 331 → 332 not taken.
✓ Branch 331 → 333 taken 2 times.
 2 if(!(env->GetCPUFlags() & CPUF_SSSE3)) // required minimum (pshufb, alignr)
7183 ✗ d.planeOptSSE2[i] = false;
7184 }
7185 // some trig.functions C only, except Sin and Cos and Atan2 and tan
7186
3/6
✓ Branch 333 → 334 taken 134 times.
✗ Branch 333 → 336 not taken.
✓ Branch 334 → 335 taken 134 times.
✗ Branch 334 → 336 not taken.
✗ Branch 335 → 336 not taken.
✓ Branch 335 → 337 taken 134 times.
 134 if (op == opAsin || op == opAcos || op == opAtan) {
7187 ✗ d.planeOptAvx2[i] = false;
7188 ✗ d.planeOptSSE2[i] = false;
7189 ✗ break;
7190 }
7191 // round, trunc, ceil: minimum of SSE4.1
7192
4/8
✓ Branch 337 → 338 taken 134 times.
✗ Branch 337 → 341 not taken.
✓ Branch 338 → 339 taken 134 times.
✗ Branch 338 → 341 not taken.
✓ Branch 339 → 340 taken 134 times.
✗ Branch 339 → 341 not taken.
✗ Branch 340 → 341 not taken.
✓ Branch 340 → 345 taken 134 times.
 134 if (op == opRound || op == opFloor || op == opCeil || op == opTrunc ) {
7193 ✗ if (!(env->GetCPUFlags() & CPUF_SSE4_1)) // required minimum (_mm_round_ps...)
7194 ✗ d.planeOptSSE2[i] = false;
7195 ✗ break;
7196 }
7197 }
7198 #endif
7199
7200 }
7201
7202 #ifdef VS_TARGET_CPU_X86
7203 // optAvx2 can only disable avx2 when available
7204
7205
3/4
✓ Branch 397 → 398 taken 56 times.
✗ Branch 397 → 417 not taken.
✓ Branch 398 → 352 taken 41 times.
✓ Branch 398 → 399 taken 15 times.
 56 for (int i = 0; i < d.vi.NumComponents(); i++) {
7206
2/2
✓ Branch 352 → 353 taken 33 times.
✓ Branch 352 → 396 taken 8 times.
 41 if (d.plane[i] == poProcess) {
7207
7208 33 const int plane_enum = plane_enums_d[i];
7209
1/2
✓ Branch 353 → 354 taken 33 times.
✗ Branch 353 → 417 not taken.
 33 int planewidth = d.vi.width >> d.vi.GetPlaneWidthSubsampling(plane_enum);
7210
1/2
✓ Branch 354 → 355 taken 33 times.
✗ Branch 354 → 417 not taken.
 33 int planeheight = d.vi.height >> d.vi.GetPlaneHeightSubsampling(plane_enum);
7211
7212
3/4
✓ Branch 355 → 356 taken 1 time.
✓ Branch 355 → 358 taken 32 times.
✓ Branch 356 → 357 taken 1 time.
✗ Branch 356 → 417 not taken.
 33 const int planewidth_real_or_lut = (lutmode == 0) ? planewidth: (1 << d.vi.BitsPerComponent());
7213 // to decide if partial chunk is left from the width at the end of the 4/8/16 pixel processing unit big main loops
7214 // when lut: fake width (x size of lut table) of the lut-init
7215
7216
3/4
✓ Branch 359 → 360 taken 6 times.
✓ Branch 359 → 377 taken 27 times.
✓ Branch 360 → 361 taken 6 times.
✗ Branch 360 → 377 not taken.
 33 if (optAvx2 && d.planeOptAvx2[i]) {
7217
7218 // avx2
7219
2/4
✓ Branch 361 → 362 taken 6 times.
✗ Branch 361 → 413 not taken.
✓ Branch 362 → 363 taken 6 times.
✗ Branch 362 → 413 not taken.
 6 ExprEvalAvx2 ExprObj(d.ops[i], d.numInputs, env->GetCPUFlags(), planewidth_real_or_lut, planeheight, optSingleMode);
7220
4/8
✓ Branch 363 → 364 taken 6 times.
✗ Branch 363 → 411 not taken.
✓ Branch 364 → 365 taken 6 times.
✗ Branch 364 → 368 not taken.
✓ Branch 366 → 367 taken 6 times.
✗ Branch 366 → 368 not taken.
✓ Branch 369 → 370 taken 6 times.
✗ Branch 369 → 375 not taken.
 6 if (ExprObj.GetCode(true) && ExprObj.GetCodeSize()) { // PF modded jitasm. true: epilog with vmovaps, and vzeroupper
7221 #ifdef VS_TARGET_OS_WINDOWS
7222 d.proc[i] = (ExprData::ProcessLineProc)VirtualAlloc(nullptr, ExprObj.GetCodeSize(), MEM_COMMIT, PAGE_EXECUTE_READWRITE);
7223 #else
7224 6 d.proc[i] = (ExprData::ProcessLineProc)mmap(nullptr, ExprObj.GetCodeSize(), PROT_READ | PROT_WRITE | PROT_EXEC, MAP_ANON | MAP_PRIVATE, 0, 0);
7225 #endif
7226
1/2
✓ Branch 373 → 374 taken 6 times.
✗ Branch 373 → 411 not taken.
 6 memcpy((void *)d.proc[i], ExprObj.GetCode(), ExprObj.GetCodeSize());
7227 }
7228 6 }
7229
3/4
✓ Branch 377 → 378 taken 7 times.
✓ Branch 377 → 395 taken 20 times.
✓ Branch 378 → 379 taken 7 times.
✗ Branch 378 → 395 not taken.
 27 else if (optSSE2 && d.planeOptSSE2[i]) {
7230 // sse2, sse4
7231
2/4
✓ Branch 379 → 380 taken 7 times.
✗ Branch 379 → 416 not taken.
✓ Branch 380 → 381 taken 7 times.
✗ Branch 380 → 416 not taken.
 7 ExprEval ExprObj(d.ops[i], d.numInputs, env->GetCPUFlags(), planewidth_real_or_lut, planeheight, optSingleMode);
7232
4/8
✓ Branch 381 → 382 taken 7 times.
✗ Branch 381 → 414 not taken.
✓ Branch 382 → 383 taken 7 times.
✗ Branch 382 → 386 not taken.
✓ Branch 384 → 385 taken 7 times.
✗ Branch 384 → 386 not taken.
✓ Branch 387 → 388 taken 7 times.
✗ Branch 387 → 393 not taken.
 7 if (ExprObj.GetCode() && ExprObj.GetCodeSize()) {
7233 #ifdef VS_TARGET_OS_WINDOWS
7234 d.proc[i] = (ExprData::ProcessLineProc)VirtualAlloc(nullptr, ExprObj.GetCodeSize(), MEM_COMMIT, PAGE_EXECUTE_READWRITE);
7235 #else
7236 7 d.proc[i] = (ExprData::ProcessLineProc)mmap(nullptr, ExprObj.GetCodeSize(), PROT_READ | PROT_WRITE | PROT_EXEC, MAP_ANON | MAP_PRIVATE, 0, 0);
7237 #endif
7238
1/2
✓ Branch 391 → 392 taken 7 times.
✗ Branch 391 → 414 not taken.
 7 memcpy((void *)d.proc[i], ExprObj.GetCode(), ExprObj.GetCodeSize());
7239 }
7240 7 }
7241
7242 } // if plane is to be processed
7243 }
7244 #ifdef VS_TARGET_OS_WINDOWS
7245 if (optSSE2)
7246 FlushInstructionCache(GetCurrentProcess(), nullptr, 0);
7247 #endif
7248 #endif
7249
7250
2/2
✓ Branch 399 → 400 taken 1 time.
✓ Branch 399 → 401 taken 14 times.
 15 if (lutmode > 0)
7251
1/2
✓ Branch 400 → 401 taken 1 time.
✗ Branch 400 → 417 not taken.
 1 calculate_lut(env);
7252
2/4
✓ Branch 402 → 403 taken 60 times.
✓ Branch 402 → 406 taken 15 times.
✗ Branch 418 → 419 not taken.
✗ Branch 418 → 422 not taken.
 75 }
7253 ✗ catch (std::runtime_error &e) {
7254 ✗ for (int i = 0; i < MAX_EXPR_INPUTS; i++)
7255 ✗ d.clips[i] = nullptr; // vsapi->freeNode(d->node[i]);
7256 ✗ std::string s = "Expr: ";
7257 ✗ s += e.what();
7258 ✗ env->ThrowError(s.c_str());
7259 ✗ return;
7260 ✗ }
7261
7262 ✗ }
7263
7264 #ifdef XP_TLS
7265 #ifdef MSVC_PURE
7266 // end of v141_xp toolset ultra slow build workaround
7267 #pragma optimize("", on)
7268 #endif
7269 #endif
7270