digits variable inaccuracy?

Gamestudio Links

Zorro Links

Zorro 2.70
by jcl. 09/29/25 09:24

ZorroGPT
by TipmyPip. 09/27/25 10:05

Plugins update
by Grant. 09/17/25 16:28

AUM Magazine

Latest Screens

3 registered members (AndrewAMD, Ayumi, NewbieZorro), 13,972 guests, and 6 spiders.

Key: Admin, Global Mod, Mod

Newest Members

krishna, DrissB, James168, Ed_Love, xtns
19168 Registered Users

Print Thread

Rate Thread

digits variable inaccuracy? #306410 01/22/10 00:06 01/22/10 00:06
Joined: Nov 2007 Posts: 1,143 United Kingdom DJBMASTER OP Serious User
DJBMASTER OP Serious User Joined: Nov 2007 Posts: 1,143 United Kingdom	Don't know if this has come up before, but following the example in the manual for 'digits' gives wrong values... Code: var x = 123456.789; ... digits(0,0,"%f",*,1,x); According to the manual this will display '123456.789' yet I keep getting '123456.789063'. I realise this is caused by the inaccuracies in converting the var to floating point, but it's still a mistake in the manual. Last edited by DJBMASTER; 01/22/10 04:18.

Re: digits variable inaccuracy? [Re: DJBMASTER] #306485 01/22/10 15:38 01/22/10 15:38
Joined: Sep 2003 Posts: 6,861 Kiel (Germany) Superku Senior Expert
Superku Senior Expert Joined: Sep 2003 Posts: 6,861 Kiel (Germany)	Quote: I realise this is caused by the inaccuracies in converting the var to floating point So there is no mistake, just limit the output to %.3f. If someone does not know why the output is 123456.789063 he will probably search/ ask for it, what will increase his knowledge (about the important fact of inaccuracies of computer maths). (Sorry for being a smart-ass.) Last edited by Superku; 01/22/10 15:39. "Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends

Re: digits variable inaccuracy? [Re: Superku] #306499 01/22/10 17:00 01/22/10 17:00
Joined: Jan 2004 Posts: 3,023 The Netherlands Helghast Expert
Helghast Expert Joined: Jan 2004 Posts: 3,023 The Netherlands	Originally Posted By: Superku Quote: I realise this is caused by the inaccuracies in converting the var to floating point So there is no mistake, just limit the output to %.3f. If someone does not know why the output is 123456.789063 he will probably search/ ask for it, what will increase his knowledge (about the important fact of inaccuracies of computer maths). (Sorry for being a smart-ass.) Yes, but then the manual still is wrong in it's example Formerly known as dennis_fantasy Portfolio - http://www.designorhea.com/ Project - http://randomchance.cherrygames.org/

Re: digits variable inaccuracy? [Re: Helghast] #306502 01/22/10 17:26 01/22/10 17:26
Joined: Sep 2003 Posts: 6,861 Kiel (Germany) Superku Senior Expert
Superku Senior Expert Joined: Sep 2003 Posts: 6,861 Kiel (Germany)	Haha ok....... I'm sorry. "Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends

Re: digits variable inaccuracy? [Re: Superku] #306836 01/25/10 10:10 01/25/10 10:10
Joined: Jul 2000 Posts: 28,024 Frankfurt jcl Chief Engineer
jcl Chief Engineer Joined: Jul 2000 Posts: 28,024 Frankfurt	Thanks, this must be %.3f - will be fixed.