file_write var array ?

Posted By: laz

file_write var array ? - 08/12/19 19:00

I need to use an array to store some data and save them to file.

But i normally not use all slots, sometimes maybe 3, 50 or 77, max 100...

How do i save the needed amount (3 in this example) of var values to file?
Code
function run() {
	
	string my_file = "Data\\FAA_TEST.csv";
	
        int  ar_used = 3;
	var my_array[100];	
	
	my_array[0] = 1000.0001;
	my_array[1] = 2000.0002;
	my_array[2] = 3000.0003;	
	
	file_delete(my_file);	
	file_write(my_file,my_array,ar_used*8);
		
	var my_read[100];
	
	file_read(my_file,my_read,10*8);
	
	int i;

	for(i=0;i<10;i++) printf("\nmy_array[%i] %f | my_read[%i] %f",i,my_array[i],i,my_read[i]);
	
	quit();
	
}
Is it correct this way or are there better ways?
Quote
my_array[0] 1000.000122 | my_read[0] 1000.000122
my_array[1] 2000.000244 | my_read[1] 2000.000244
my_array[2] 3000.000244 | my_read[2] 3000.000244
Later i want to read them, why does this 1000.0001>22< happen?

Thanks!
Posted By: laz

Re: file_write var array ? - 08/12/19 21:43

I think i'll use the data functions for that wink

https://manual.zorro-project.com/data.htm

But, why does this 1000.0001>22< happen?
Posted By: jcl

Re: file_write var array ? - 08/13/19 15:24

Because computer math is slightly different than the math at school, due to the finite precision of variables.

https://en.wikipedia.org/wiki/Floating-point_arithmetic
Posted By: laz

Re: file_write var array ? - 08/14/19 02:34

Autsch... Thanks jcl crazy

Sorry but nobody has teached me computer math. I've taught myself what I know about programming, and sometimes I just miss things.

NEW MINIMAL EXAMPLE
Code
function run() {
		
	int i;
	var my_array[10];
	
	my_array[0] = 1000.0111;
	my_array[1] = 2000.0022;
	my_array[2] = 3000.0003;	
	
	watch("test0 = ",(1000.0111 - my_array[0]) == 0.0);
	watch("test1 = ",(2000.0022 - my_array[1]) == 0.0);
	watch("test2 = ",(3000.0003 - my_array[2]) == 0.0);
	
	for(i=0;i<4;i++) printf("\nmy_array[%i] %f",i,my_array[i]);
	
	quit();
	
}
Quote
file-append-array2 compiling...........
test0 = 1
test1 = 1
test2 = 1
my_array[0] 1000.011108
my_array[1] 2000.002197
my_array[2] 3000.000244
my_array[3] 0.000000
Quit
test0,1,2 are TRUE. I would think, that my printing is wrong - but what is wrong?

If i round 3000.000244 to 4 digits, it is 3000.0002 and not 3000.0003, why the initial zero (0) of my_array[3] stays unchanged and the 3000.0003 changes to 3000.000244?

Even if I ask a simple or stupid question, it would be nice if someone answers ...

Thanks for your help...
Posted By: Grat

Re: file_write var array ? - 08/17/19 11:24

I make small modification
Code
function main() {
		
	int i;
	//var *my_array=zalloc(4*sizeof(var));
	var my_array[4]={0.0,0.0,0.0,0.0};
  
       for(i=0;i<4;i++) printf("\nmy_array[%i] %.5f",i,my_array[i]);
	
	my_array[0] =(var) 1000.0111000;
	my_array[1] =(var) 2000.0022000;
	my_array[2] =(var) 3000.0003000;	
	
	var n=0;
	for(i=0;i<4;i++){ 
            printf("\nmy_array[%i] %.8f",i,my_array[i]);
            n+=my_array[i];
       }
       printf("\n%f ",n);
       for(i=0;i<4;i++) printf("\nmy_array[%i] %.8g",i,my_array[i]);
	
}


output...

Quote

my_array[1] 0.00000
my_array[2] 0.00000
my_array[3] 0.00000
my_array[0] 1000.01110840
my_array[1] 2000.00219727
my_array[2] 3000.00024414
my_array[3] 0.00000000
6000.013550
my_array[0] 1000.0111
my_array[1] 2000.0022
my_array[2] 3000.0002
my_array[3] 0



So, i thinking something is a wrong.

1000.0111+2000.0022+3000.0002 = 6000.013600

but no 6000.013550

Posted By: Grat

Re: file_write var array ? - 08/17/19 11:34

in the python:
Code
my_array = [1000.0111000, 2000.0022000, 3000.0003000]
n=0
for x in my_array:
    print ("{:.9f}".format(x),)
    n+=x
print ("{:.9f}".format(n),)


is output:
1000.011100000
2000.002200000
3000.000300000
6000.013600000
Posted By: Grat

Re: file_write var array ? - 08/17/19 11:47

Ok, next test:

Code
function main() {
		
  double d3=100.0003;
  double d2=1.0002;
  printf("\n%f+ %f= %f",d3,d2,d3+d2);

  var v3=100.0003;
  var v2=1.0002;
  printf("\n%f+ %f= %f",v3,v2,v3+v2);

  float f3=100.0003;
  float f2=1.0002;
  printf("\n%f+ %f= %f",f3,f2,f3+f2);
  
}


with output:


test1 compiling................ ok

100.000298+ 1.000200= 101.000498
100.000298+ 1.000200= 101.000498
0.007825+ -2.000000= -26815622246983973000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.000000
Posted By: laz

Re: file_write var array ? - 08/19/19 12:33

Thanks grat for testing, there is a small typo in your first post:

Quote
1000.0111+2000.0022+3000.0002 = 6000.013600

1 000.0111 + 2 000.0022 + 3 000.0003 = 6 000.0136

What are we missing jcl?
Posted By: jcl

Re: file_write var array ? - 08/20/19 09:03

I don't even know what the problem is, let alone what you are missing.
Posted By: laz

Re: file_write var array ? - 08/20/19 12:54

The question is why Zorro shows changed values:

my_array[0] 1000.01110840
my_array[1] 2000.00219727
my_array[2] 3000.00024414

instead of

my_array[0] = 1000.0111;
my_array[1] = 2000.0022;
my_array[2] = 3000.0003;

Your answer was :

Quote
Because computer math is slightly different than the math at school, due to the finite precision of variables.

https://en.wikipedia.org/wiki/Floating-point_arithmetic


I don't understand that, can you please explain?
Posted By: AndrewAMD

Re: file_write var array ? - 08/20/19 13:24

A var is actually a double. The characteristics of computer mathematics affect how doubles add and subtract. This has nothing to do with Zorro.
Posted By: laz

Re: file_write var array ? - 08/20/19 17:18

It means (if i understand right) that if i do this:

my_array[2] = 3000.0003;

The internal value of my_array[2] is 3000.00024414 ?

And that is why (3000.0003 (real 3000.00024414) - my_array[2] (real 3000.00024414)) == 0 ?

That helped me a bit:
https://www.youtube.com/watch?v=PZRI1IfStY0
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

I think I got it (in parts), I have to admit that I have some knowledge gaps in this area. I keep reading about it, and while I'm doing this i can see why i never had problems with it. I always set a precision (like in R options(digits = 12)) and round to the accuracy i need, and i normally never do this if (double == double) without rounding.

Code
> R-CODE
options(digits = 12)
> a<-3000.0003
> a
[1] 3000.0003
> options(digits = 15)
> a<-3000.0003
> a
[1] 3000.0003
> options(digits = 16)
> a<-3000.0003
> a
[1] 3000.0003
> options(digits = 17)
> a<-3000.0003
> a
[1] 3000.0003000000002
> options(digits = 18)
> a<-3000.0003
> a
[1] 3000.00030000000015
> options(digits = 20)
> a<-3000.0003
> a
[1] 3000.0003000000001521

Why is R not adding anything until digits >= 17? Or better: Why is the value in C-Lite 3000.00024414 and in R 3000.0003>>00000000<< up to digits = 16?

How can i influence this in C-lite? Because at this point rounding(3000.00024414, to 4 digits) will result in 3000.0002, not 3000.0003 - and in R that works fine?

Many thanks for helping me with that basic stuff!
Posted By: laz

Re: file_write var array ? - 08/29/19 12:27

Guys, I'm really sorry - but I still need help with that crazy...

I read all the links (and much more):

https://en.wikipedia.org/wiki/Floating-point_arithmetic
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
https://opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=467836
https://zorro-project.com/manual/en/aarray.htm
https://zorro-project.com/manual/en/format.htm

Code
var t1 = 0.1;

file_append(tfn,strf("%i,%.14f,%.14f,%.14f\n",Bar,t1,(var)t1,(double)t1),0);

result: 10,0.10000000149012,0.10000000149012,0.10000000149012

https://zorro-project.com/manual/en/aarray.htm
Type double, var | Size (bytes) 8 | Digits ~14

http://www.binaryconvert.com/result_float.html?decimal=048046049 | float(.1) | 0.100000001490116119384765625
http://www.binaryconvert.com/result_double.html?decimal=048046049 | double(.1) | 0.1000000000000000055511151231257827021181583404541015625

Why does file_append/strf write var/double as floats?

Quote
As the printf class of functions are variadic (denoted by the elipsis „…“ in the function signature) the default argument promotions take place. Finally the call printf("float = %f\n", f) will after promotion look like printf("float = %f\n", (double)f) and printf can safely assume that every %f, %g, and %e denotes a parameter of type double. https://blogs.fau.de/wittmann/2013/11/c-default-argument-promotions/

Why does printf print doubles as a float?

"%f Floating-point number (var, double) " https://zorro-project.com/manual/en/format.htm

"The printf function does not automatically promote float to double." https://manual.zorro-project.com/printf.htm

But double to float?
Quote
The %f format specifier for the printf/scanf class of functions denotes a variable of type float which consumes 32 bits. In the example above d is of type double which uses 64 bits and actually requires %lf. The provided argument d (64 bits) would be interpreted as a variable of type float (32 bits). https://blogs.fau.de/wittmann/2013/11/c-default-argument-promotions/

%lf does not exist in clite or does it?

Quote
Rounding errors occur when a number can’t be stored precisely. This can happen even with simple numbers, like 0.1. Therefore, rounding errors can, and do, happen all the time.
A corollary of this rule is: never use floating point numbers for financial or currency data. https://www.learncpp.com/cpp-tutorial/floating-point-numbers

I'm clearly missing something here and i don't get it...

Many thanks!
Posted By: JamesHH

Re: file_write var array ? - 08/31/19 01:33

Actually, I noticed something very similar the other day: I was working with type var and they only seemed to have float precision instead of double, which is concerning.

I will test some more when I have time.

You can do arithmetic operations instead of printf to test the accuracy. For example, you can subtract close values, like 1.00000001 - 1.0,
and check the result which would be 0 for type float but >0 for type double.
Posted By: laz

Re: file_write var array ? - 08/31/19 11:42

Thanks for joining james...

Either I have overlooked it or it is nowhere mentioned that data written/printed will be transformed and thereby lose precision. The R-bridge also sends data (double) different from those written by Zorro (float) in text files.I have now read more intensively about the topic and unfortunately there are many pitfalls, I would find it easier and better, if we can determine the precision.

Example:% f for floats% lf or something for doubles

How to print/write data with more precision?

I can't use printf, strf, file functions and so on or is that wrong - or is it possible and I'm just too stupid?
Posted By: JamesHH

Re: file_write var array ? - 09/04/19 05:18

There seems to be two different possibilities here. 1) The value in memory is double precision, but is being printed as float. This would be strange. printf has been part of C for decades and the format string is standard. I can't remember, but you can look up how to print a specified number of digits after the decimal point. Something like %f.12 or %f12, I can't recall ...

2) The value in memory is only floating point precision.

Have you ruled out 2?
Posted By: laz

Re: file_write var array ? - 09/05/19 22:19

1) The value in memory is double precision, but is being printed as float.

^^^ I think that's it...

"This would be strange. printf has been part of C for decades and the format string is standard."

That was the reason why i was confused - i did not expect that - apart from my knowledge gaps in floating point arithmetic.

The manual says: "For printing float variables with the %f placeholder, typecast them to (var) or (double) (f.i. printf("Profit: %.2f",(var)TradeProfit);). "

But as you can see in my examples, typecasting to (var) or (double) does not work.

Code
var t1 = 0.1;

file_append("Data/test-write.csv",strf("%i,%.14f,%.14f,%.14f\n",Bar,t1,(var)t1,(double)t1),0);

result: 10,0.10000000149012,0.10000000149012,0.10000000149012

If i printf a (var) I'll expect a (var==double) to be printed, precision depending on the format string like "%.8f"...

Even writing into a csv with double precision is not possible at the moment - or am I wrong?

Maybe jcl can explain it in easy words, what's correct and why it is like it is?
Posted By: JamesHH

Re: file_write var array ? - 09/06/19 16:47

Originally Posted by laz
1) The value in memory is double precision, but is being printed as float.

^^^ I think that's it...


Well, as far as I can tell you have not confirmed that this is the case.

If t1 in your code snippet was a float, then your output would be exactly as expected.

What happens if you replace

Code
var t1 = 0.1;


with

Code
float t1 = 0.1;
?
Posted By: jcl

Re: file_write var array ? - 09/09/19 09:19

The types var and double are identical. They have 64 bit precision, while float has 32 bit precision.
Posted By: JamesHH

Re: file_write var array ? - 09/11/19 16:47

Originally Posted by jcl
The types var and double are identical. They have 64 bit precision, while float has 32 bit precision.


They are identical, but in Zorro scripts double is *not* an IEEE754 Double precision 64-bit. The following script demonstrates a double that is being represented the same as an IEEE754 32-bit float:

function main()
{
double x = 1.00000001;

if(x == 1.0) {
printf("\nx is float");
} else {
printf("\nx is double");
}
}

Can you please clarify what is going on? This is very serious!!
Posted By: AndrewAMD

Re: file_write var array ? - 09/11/19 17:19

<snip>
Posted By: JamesHH

Re: file_write var array ? - 09/11/19 17:39

Umm, you've just proved my point.

Yes, var is typedef'd as double, but that's not the issue.

Your output shows that, in Zorro, var/double is behaving just like a 32-bit float:

float representation of 0.123456789012345678901234567890

That's called an MRE, not production code. Which branch do you think should execute?
Posted By: Petra

Re: file_write var array ? - 09/13/19 17:26

You can easily see with a tiny script that var behaves like a 64 bit double. But number representation and variable precision are not the same. Variables have 15 decimals, numbers written in your script have probably less.
Posted By: JamesHH

Re: file_write var array ? - 09/13/19 20:18

Originally Posted by Petra
You can easily see with a tiny script that var behaves like a 64 bit double. But number representation and variable precision are not the same. Variables have 15 decimals, numbers written in your script have probably less.


No, my script demonstrates that var/double behaves like a 32-bit precision float.

Did you see the one above where 1.00000001 == 1.0?

Here is another script showing that type "double" is 32-bit precision in Zorro:

function main()
{
double x = 0.123456789012345;

printf("\nx = %.15f", x);
}

Output:

x = 0.123456791043282

The output is the float representation of the number 0.123456789012345: float 32-bit representation of 0.123456789012345
Posted By: Petra

Re: file_write var array ? - 09/13/19 20:25

That script wont work, youre still confusing number representation with precision.

Standard test program for variable precision:

Code
void main()
{
	double d = 1000000000000000.;  // 15 zeros -> 64 bits
	d = 1. + 1./d;
	printf("\n %.15f",d);
}

laugh
Posted By: JamesHH

Re: file_write var array ? - 09/13/19 22:49

Interesting!

So why is x being set to 1.0 in the following script?

Code

function main()
{
	double x = 1.00000001;
	printf("\n %.15f", x);
}

Posted By: JamesHH

Re: file_write var array ? - 09/13/19 22:59

Oh, so you are saying that *constants* do not have the same precision as variables?
Posted By: jcl

Re: file_write var array ? - 09/14/19 08:31

Numbers in scripts are interpreted as either int or float, depending on decimal point. This must be kept in mind when assigning very large or very small constants or constant expressions to a variable. You can find details in the manual under "Variables". This has nothing to do with the bit size of a double.
Posted By: AndrewAMD

Re: file_write var array ? - 09/15/19 12:17

As a workaround, you can enter a high-precision variable as a string and then parse it.

Code
#include <default.c>
void main() {
	var v = strvar("0.123456789012345678901234567890",0,0.0);
	printf("\nv: %0.30f",v);
	v *= 10000000000.;
	printf("\nv: %0.30f",v);
	printf("\ndone.");
}
/*  output:
v: 0.123456789012345680000000000000
v: 1234567890.123456700000000000000000000000
done.
*/


Here's the original script, fixed:
Code
#include <default.c>
void run() {
	string my_file = "Data\\FAA_TEST.csv";

	int  ar_used = 3;
	var my_array[100];	

	my_array[0] = strvar("1000.0001",0,0);
	my_array[1] = strvar("2000.0002",0,0);
	my_array[2] = strvar("3000.0003",0,0);

	file_delete(my_file);	
	file_write(my_file,my_array,ar_used*8);
		
	var my_read[100];

	file_read(my_file,my_read,10*8);

	int i;

	for(i=0;i<10;i++) printf("\nmy_array[%i] %f | my_read[%i] %f",i,my_array[i],i,my_read[i]);

	quit();
}
/*  OUTPUT:
	float_test compiling...........
	my_array[0] 1000.000100 | my_read[0] 1000.000100
	my_array[1] 2000.000200 | my_read[1] 2000.000200
	my_array[2] 3000.000300 | my_read[2] 3000.000300
	my_array[3] 0.000000 | my_read[3] 0.000000
	my_array[4] 0.000000 | my_read[4] 0.000000
	my_array[5] 0.000000 | my_read[5] 0.000000
	my_array[6] 0.000000 | my_read[6] 0.000000
	my_array[7] 0.000000 | my_read[7] 0.000000
	my_array[8] 0.000000 | my_read[8] 0.000000
	my_array[9] 0.000000 | my_read[9] 0.000000
	Quit
*/


Or you can write your script in VC++, so you will not need to put up with the eccentricities of the Lite-C compiler. laugh
Posted By: JamesHH

Re: file_write var array ? - 09/16/19 00:32

Thanks for the suggestions.

Yes, but this requires Zorro-S if I understood the manual. I should have Zorro-S considering how much I use it.
Posted By: jcl

Re: file_write var array ? - 09/16/19 10:18

VC++ requires Zorro S, but there is certainly also a solution for your problem with the free version. I only need to understand what you want to do. For what purpose do you have to write 15-digit numbers in your script?
Posted By: JamesHH

Re: file_write var array ? - 09/16/19 15:16

Originally Posted by jcl
VC++ requires Zorro S, but there is certainly also a solution for your problem with the free version. I only need to understand what you want to do. For what purpose do you have to write 15-digit numbers in your script?


I don't tongue My concern was that something was wrong and doubles were only 32-bit, based on my test scripts loading from a constant.

Still, I think the ability to run scripts through the MSVS debugger is enough of a reason to justify upgrading to Zorro S.
© 2019 lite-C Forums