Merge lp:~openerp-dev/openobject-server/trunk-float-rounding-odo into lp:openobject-server
- trunk-float-rounding-odo
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 3911 |
Proposed branch: | lp:~openerp-dev/openobject-server/trunk-float-rounding-odo |
Merge into: | lp:openobject-server |
Diff against target: |
508 lines (+406/-18) 6 files modified
openerp/addons/base/res/res_currency.py (+42/-8) openerp/addons/base/test/base_test.yml (+160/-0) openerp/osv/fields.py (+16/-9) openerp/tools/__init__.py (+1/-0) openerp/tools/float_utils.py (+186/-0) openerp/tools/misc.py (+1/-1) |
To merge this branch: | bzr merge lp:~openerp-dev/openobject-server/trunk-float-rounding-odo |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Vo Minh Thu (community) | Approve | ||
Lorenzo Battistini (community) | Abstain | ||
Review via email: mp+82206@code.launchpad.net |
Commit message
Description of the change
- Extracted res.currency's float computation methods to tools, so they can be reused cleanly for other floating-point operations.
- Also added a compare() method to allow for easy comparison of 2 float values, as suggested by Ferdinand on bug 865387
- Added tests and docstrings to explain the logic.
Watch out for compare(), because compare(
The utility methods used in the YAML tests take redundant arguments due to idiotic namespace issues, feel free to suggest alternatives.
Ferdinand (office-chricar) wrote : | # |
Xavier (Open ERP) (xmo-deactivatedaccount) wrote : | # |
That's kind-of nitpicky, but in `float_is_zero` I think `rounding_factor` should be renamed `epsilon`, as that's the term usually used for floating-point relative errors: http://
Cloves Almeida (cjalmeida) wrote : | # |
"misc.py" is over 1500 lines already. Since this is to be used everywhere, maybe a "tools/float.py" is a better approach?
Ferdinand (office-chricar) wrote : | # |
Have checked with data from
https:/
and it works nicely
I attach a patched for base_test.yml with extended tests and a bit more verboseitiy
here
https:/
Lorenzo Battistini (elbati) wrote : | # |
Hello Olivier,
I think I found an issue:
float_round(
17.35
Olivier Dony (Odoo) (odo-openerp) wrote : | # |
> I think I found an issue:
>
> float_round(
> 17.35
That's the expected result as far as I can see. Perhaps you meant to test with precision_
>>> from decimal import Decimal, ROUND_HALF_UP
>>> Decimal(
Decimal('17.35')
>>> Decimal(
Decimal('17.4')
>>>
Anyway the code still needs to be updated to cover more tricky half-up cases, as discussed on bug 882036.
Olivier Dony (Odoo) (odo-openerp) wrote : | # |
> "misc.py" is over 1500 lines already. Since this is to be used everywhere,
> maybe a "tools/float.py" is a better approach?
Good point, moved it to tools/float_
Lorenzo Battistini (elbati) wrote : | # |
>>
>> float_round(
>> 17.35
>
> That's the expected result as far as I can see.
Sorry Olivier, you are right. I wrote it too quickly and had another
thing in mind.
Lorenzo Battistini (elbati) : | # |
Olivier Dony (Odoo) (odo-openerp) wrote : | # |
I think I'm done with the final changes here:
- tie-breaking epsilon has been implemented similarly to what was discussed on bug bug 882036 in order to perform proper HALF-UP rounding
- an additional float_repr method was added in float_utils to render a float to a str, as the default str() and repr() do not do what we need. repr() will not round appropriately, and str() will round too eagerly when there are more than 12 significant digits, for some reason.
- the ORM has been updated to properly apply rounding before persisting a value (this is really what bug 882036 was about). It passes the float values to postgres using float_repr, to avoid any precision loss.
- YAML tests updated to include thousands of cases from low to high magnitudes + a test of float round-trip via the database too
Vo Minh Thu (thu) wrote : | # |
I haven't spotted anything obviously stupid.
Tests could be moved to openerp.tests but this can wait until the runbot really run those.
Preview Diff
1 | === modified file 'openerp/addons/base/res/res_currency.py' | |||
2 | --- openerp/addons/base/res/res_currency.py 2011-10-11 16:34:35 +0000 | |||
3 | +++ openerp/addons/base/res/res_currency.py 2011-12-21 01:14:24 +0000 | |||
4 | @@ -24,7 +24,7 @@ | |||
5 | 24 | from osv import fields, osv | 24 | from osv import fields, osv |
6 | 25 | import tools | 25 | import tools |
7 | 26 | 26 | ||
9 | 27 | from tools.misc import currency | 27 | from tools import float_round, float_is_zero, float_compare |
10 | 28 | from tools.translate import _ | 28 | from tools.translate import _ |
11 | 29 | 29 | ||
12 | 30 | CURRENCY_DISPLAY_PATTERN = re.compile(r'(\w+)\s*(?:\((.*)\))?') | 30 | CURRENCY_DISPLAY_PATTERN = re.compile(r'(\w+)\s*(?:\((.*)\))?') |
13 | @@ -127,15 +127,49 @@ | |||
14 | 127 | return [(x['id'], tools.ustr(x['name']) + (x['symbol'] and (' (' + tools.ustr(x['symbol']) + ')') or '')) for x in reads] | 127 | return [(x['id'], tools.ustr(x['name']) + (x['symbol'] and (' (' + tools.ustr(x['symbol']) + ')') or '')) for x in reads] |
15 | 128 | 128 | ||
16 | 129 | def round(self, cr, uid, currency, amount): | 129 | def round(self, cr, uid, currency, amount): |
23 | 130 | if currency.rounding == 0: | 130 | """Return ``amount`` rounded according to ``currency``'s |
24 | 131 | return 0.0 | 131 | rounding rules. |
25 | 132 | else: | 132 | |
26 | 133 | # /!\ First member below must be rounded to full unit! | 133 | :param browse_record currency: currency for which we are rounding |
27 | 134 | # Do not pass a rounding digits value to round() | 134 | :param float amount: the amount to round |
28 | 135 | return round(amount / currency.rounding) * currency.rounding | 135 | :return: rounded float |
29 | 136 | """ | ||
30 | 137 | return float_round(amount, precision_rounding=currency.rounding) | ||
31 | 138 | |||
32 | 139 | def compare_amounts(self, cr, uid, currency, amount1, amount2): | ||
33 | 140 | """Compare ``amount1`` and ``amount2`` after rounding them according to the | ||
34 | 141 | given currency's precision.. | ||
35 | 142 | An amount is considered lower/greater than another amount if their rounded | ||
36 | 143 | value is different. This is not the same as having a non-zero difference! | ||
37 | 144 | |||
38 | 145 | For example 1.432 and 1.431 are equal at 2 digits precision, | ||
39 | 146 | so this method would return 0. | ||
40 | 147 | However 0.006 and 0.002 are considered different (returns 1) because | ||
41 | 148 | they respectively round to 0.01 and 0.0, even though | ||
42 | 149 | 0.006-0.002 = 0.004 which would be considered zero at 2 digits precision. | ||
43 | 150 | |||
44 | 151 | :param browse_record currency: currency for which we are rounding | ||
45 | 152 | :param float amount1: first amount to compare | ||
46 | 153 | :param float amount2: second amount to compare | ||
47 | 154 | :return: (resp.) -1, 0 or 1, if ``amount1`` is (resp.) lower than, | ||
48 | 155 | equal to, or greater than ``amount2``, according to | ||
49 | 156 | ``currency``'s rounding. | ||
50 | 157 | """ | ||
51 | 158 | return float_compare(amount1, amount2, precision_rounding=currency.rounding) | ||
52 | 136 | 159 | ||
53 | 137 | def is_zero(self, cr, uid, currency, amount): | 160 | def is_zero(self, cr, uid, currency, amount): |
55 | 138 | return abs(self.round(cr, uid, currency, amount)) < currency.rounding | 161 | """Returns true if ``amount`` is small enough to be treated as |
56 | 162 | zero according to ``currency``'s rounding rules. | ||
57 | 163 | |||
58 | 164 | Warning: ``is_zero(amount1-amount2)`` is not always equivalent to | ||
59 | 165 | ``compare_amounts(amount1,amount2) == 0``, as the former will round after | ||
60 | 166 | computing the difference, while the latter will round before, giving | ||
61 | 167 | different results for e.g. 0.006 and 0.002 at 2 digits precision. | ||
62 | 168 | |||
63 | 169 | :param browse_record currency: currency for which we are rounding | ||
64 | 170 | :param float amount: amount to compare with currency's zero | ||
65 | 171 | """ | ||
66 | 172 | return float_is_zero(amount, precision_rounding=currency.rounding) | ||
67 | 139 | 173 | ||
68 | 140 | def _get_conversion_rate(self, cr, uid, from_currency, to_currency, context=None): | 174 | def _get_conversion_rate(self, cr, uid, from_currency, to_currency, context=None): |
69 | 141 | if context is None: | 175 | if context is None: |
70 | 142 | 176 | ||
71 | === modified file 'openerp/addons/base/test/base_test.yml' | |||
72 | --- openerp/addons/base/test/base_test.yml 2011-07-28 08:27:52 +0000 | |||
73 | +++ openerp/addons/base/test/base_test.yml 2011-12-21 01:14:24 +0000 | |||
74 | @@ -144,3 +144,163 @@ | |||
75 | 144 | !python {model: res.partner.category}: | | 144 | !python {model: res.partner.category}: | |
76 | 145 | self.pool._init = True | 145 | self.pool._init = True |
77 | 146 | 146 | ||
78 | 147 | - | ||
79 | 148 | "Float precision tests: verify that float rounding methods are working correctly via res.currency" | ||
80 | 149 | - | ||
81 | 150 | !python {model: res.currency}: | | ||
82 | 151 | from tools import float_repr | ||
83 | 152 | from math import log10 | ||
84 | 153 | currency = self.browse(cr, uid, ref('base.EUR')) | ||
85 | 154 | def try_round(amount, expected, self=self, cr=cr, currency=currency, float_repr=float_repr, | ||
86 | 155 | log10=log10): | ||
87 | 156 | digits = max(0,-int(log10(currency.rounding))) | ||
88 | 157 | result = float_repr(self.round(cr, 1, currency, amount), precision_digits=digits) | ||
89 | 158 | assert result == expected, 'Rounding error: got %s, expected %s' % (result, expected) | ||
90 | 159 | try_round(2.674,'2.67') | ||
91 | 160 | try_round(2.675,'2.68') # in Python 2.7.2, round(2.675,2) gives 2.67 | ||
92 | 161 | try_round(-2.675,'-2.68') # in Python 2.7.2, round(2.675,2) gives 2.67 | ||
93 | 162 | try_round(0.001,'0.00') | ||
94 | 163 | try_round(-0.001,'-0.00') | ||
95 | 164 | try_round(0.0049,'0.00') # 0.0049 is closer to 0 than to 0.01, so should round down | ||
96 | 165 | try_round(0.005,'0.01') # the rule is to round half away from zero | ||
97 | 166 | try_round(-0.005,'-0.01') # the rule is to round half away from zero | ||
98 | 167 | |||
99 | 168 | def try_zero(amount, expected, self=self, cr=cr, currency=currency): | ||
100 | 169 | assert self.is_zero(cr, 1, currency, amount) == expected, "Rounding error: %s should be zero!" % amount | ||
101 | 170 | try_zero(0.01, False) | ||
102 | 171 | try_zero(-0.01, False) | ||
103 | 172 | try_zero(0.001, True) | ||
104 | 173 | try_zero(-0.001, True) | ||
105 | 174 | try_zero(0.0046, True) | ||
106 | 175 | try_zero(-0.0046, True) | ||
107 | 176 | try_zero(2.68-2.675, False) # 2.68 - 2.675 = 0.005 -> rounds to 0.01 | ||
108 | 177 | try_zero(2.68-2.676, True) # 2.68 - 2.675 = 0.004 -> rounds to 0.0 | ||
109 | 178 | try_zero(2.676-2.68, True) # 2.675 - 2.68 = -0.004 -> rounds to -0.0 | ||
110 | 179 | try_zero(2.675-2.68, False) # 2.675 - 2.68 = -0.005 -> rounds to -0.01 | ||
111 | 180 | |||
112 | 181 | def try_compare(amount1, amount2, expected, self=self, cr=cr, currency=currency): | ||
113 | 182 | assert self.compare_amounts(cr, 1, currency, amount1, amount2) == expected, \ | ||
114 | 183 | "Rounding error, compare_amounts(%s,%s) should be %s" % (amount1, amount2, expected) | ||
115 | 184 | try_compare(0.001, 0.001, 0) | ||
116 | 185 | try_compare(-0.001, -0.001, 0) | ||
117 | 186 | try_compare(0.001, 0.002, 0) | ||
118 | 187 | try_compare(-0.001, -0.002, 0) | ||
119 | 188 | try_compare(2.675, 2.68, 0) | ||
120 | 189 | try_compare(2.676, 2.68, 0) | ||
121 | 190 | try_compare(-2.676, -2.68, 0) | ||
122 | 191 | try_compare(2.674, 2.68, -1) | ||
123 | 192 | try_compare(-2.674, -2.68, 1) | ||
124 | 193 | try_compare(3, 2.68, 1) | ||
125 | 194 | try_compare(-3, -2.68, -1) | ||
126 | 195 | try_compare(0.01, 0, 1) | ||
127 | 196 | try_compare(-0.01, 0, -1) | ||
128 | 197 | |||
129 | 198 | - | ||
130 | 199 | "Float precision tests: verify that float rounding methods are working correctly via tools" | ||
131 | 200 | - | ||
132 | 201 | !python {model: res.currency}: | | ||
133 | 202 | from tools import float_compare, float_is_zero, float_round, float_repr | ||
134 | 203 | def try_round(amount, expected, precision_digits=3, float_round=float_round, float_repr=float_repr): | ||
135 | 204 | result = float_repr(float_round(amount, precision_digits=precision_digits), | ||
136 | 205 | precision_digits=precision_digits) | ||
137 | 206 | assert result == expected, 'Rounding error: got %s, expected %s' % (result, expected) | ||
138 | 207 | try_round(2.6745, '2.675') | ||
139 | 208 | try_round(-2.6745, '-2.675') | ||
140 | 209 | try_round(2.6744, '2.674') | ||
141 | 210 | try_round(-2.6744, '-2.674') | ||
142 | 211 | try_round(0.0004, '0.000') | ||
143 | 212 | try_round(-0.0004, '-0.000') | ||
144 | 213 | try_round(357.4555, '357.456') | ||
145 | 214 | try_round(-357.4555, '-357.456') | ||
146 | 215 | try_round(457.4554, '457.455') | ||
147 | 216 | try_round(-457.4554, '-457.455') | ||
148 | 217 | |||
149 | 218 | # Extended float range test, inspired by Cloves Almeida's test on bug #882036. | ||
150 | 219 | fractions = [.0, .015, .01499, .675, .67499, .4555, .4555, .45555] | ||
151 | 220 | expecteds = ['.00', '.02', '.01', '.68', '.67', '.46', '.456', '.4556'] | ||
152 | 221 | precisions = [2, 2, 2, 2, 2, 2, 3, 4] | ||
153 | 222 | # Note: max precision for double floats is 53 bits of precision or | ||
154 | 223 | # 17 significant decimal digits | ||
155 | 224 | for magnitude in range(7): | ||
156 | 225 | for i in xrange(len(fractions)): | ||
157 | 226 | frac, exp, prec = fractions[i], expecteds[i], precisions[i] | ||
158 | 227 | for sign in [-1,1]: | ||
159 | 228 | for x in xrange(0,10000,97): | ||
160 | 229 | n = x * 10**magnitude | ||
161 | 230 | f = sign * (n + frac) | ||
162 | 231 | f_exp = ('-' if f != 0 and sign == -1 else '') + str(n) + exp | ||
163 | 232 | try_round(f, f_exp, precision_digits=prec) | ||
164 | 233 | |||
165 | 234 | |||
166 | 235 | def try_zero(amount, expected, float_is_zero=float_is_zero): | ||
167 | 236 | assert float_is_zero(amount, precision_digits=3) == expected, "Rounding error: %s should be zero!" % amount | ||
168 | 237 | try_zero(0.0002, True) | ||
169 | 238 | try_zero(-0.0002, True) | ||
170 | 239 | try_zero(0.00034, True) | ||
171 | 240 | try_zero(0.0005, False) | ||
172 | 241 | try_zero(-0.0005, False) | ||
173 | 242 | try_zero(0.0008, False) | ||
174 | 243 | try_zero(-0.0008, False) | ||
175 | 244 | |||
176 | 245 | def try_compare(amount1, amount2, expected, float_compare=float_compare): | ||
177 | 246 | assert float_compare(amount1, amount2, precision_digits=3) == expected, \ | ||
178 | 247 | "Rounding error, compare_amounts(%s,%s) should be %s" % (amount1, amount2, expected) | ||
179 | 248 | try_compare(0.0003, 0.0004, 0) | ||
180 | 249 | try_compare(-0.0003, -0.0004, 0) | ||
181 | 250 | try_compare(0.0002, 0.0005, -1) | ||
182 | 251 | try_compare(-0.0002, -0.0005, 1) | ||
183 | 252 | try_compare(0.0009, 0.0004, 1) | ||
184 | 253 | try_compare(-0.0009, -0.0004, -1) | ||
185 | 254 | try_compare(557.4555, 557.4556, 0) | ||
186 | 255 | try_compare(-557.4555, -557.4556, 0) | ||
187 | 256 | try_compare(657.4444, 657.445, -1) | ||
188 | 257 | try_compare(-657.4444, -657.445, 1) | ||
189 | 258 | |||
190 | 259 | # Rounding to unusual rounding units (e.g. coin values) | ||
191 | 260 | def try_round(amount, expected, precision_rounding=None, float_round=float_round, float_repr=float_repr): | ||
192 | 261 | result = float_repr(float_round(amount, precision_rounding=precision_rounding), | ||
193 | 262 | precision_digits=2) | ||
194 | 263 | assert result == expected, 'Rounding error: got %s, expected %s' % (result, expected) | ||
195 | 264 | try_round(-457.4554, '-457.45', precision_rounding=0.05) | ||
196 | 265 | try_round(457.444, '457.50', precision_rounding=0.5) | ||
197 | 266 | try_round(457.3, '455.00', precision_rounding=5) | ||
198 | 267 | try_round(457.5, '460.00', precision_rounding=5) | ||
199 | 268 | try_round(457.1, '456.00', precision_rounding=3) | ||
200 | 269 | |||
201 | 270 | - | ||
202 | 271 | "Float precision tests: check that proper rounding is performed for float persistence" | ||
203 | 272 | - | ||
204 | 273 | !python {model: res.currency}: | | ||
205 | 274 | currency = self.browse(cr, uid, ref('base.EUR')) | ||
206 | 275 | res_currency_rate = self.pool.get('res.currency.rate') | ||
207 | 276 | from tools import float_compare, float_is_zero, float_round, float_repr | ||
208 | 277 | def try_roundtrip(value, expected, self=self, cr=cr, currency=currency, | ||
209 | 278 | res_currency_rate=res_currency_rate): | ||
210 | 279 | rate_id = res_currency_rate.create(cr, 1, {'name':'2000-01-01', | ||
211 | 280 | 'rate': value, | ||
212 | 281 | 'currency_id': currency.id}) | ||
213 | 282 | rate = res_currency_rate.read(cr, 1, rate_id, ['rate'])['rate'] | ||
214 | 283 | assert rate == expected, 'Roundtrip error: got %s back from db, expected %s' % (rate, expected) | ||
215 | 284 | # res.currency.rate uses 6 digits of precision by default | ||
216 | 285 | try_roundtrip(2.6748955, 2.674896) | ||
217 | 286 | try_roundtrip(-2.6748955, -2.674896) | ||
218 | 287 | try_roundtrip(10000.999999, 10000.999999) | ||
219 | 288 | try_roundtrip(-10000.999999, -10000.999999) | ||
220 | 289 | |||
221 | 290 | - | ||
222 | 291 | "Float precision tests: verify that invalid parameters are forbidden" | ||
223 | 292 | - | ||
224 | 293 | !python {model: res.currency}: | | ||
225 | 294 | from tools import float_compare, float_is_zero, float_round | ||
226 | 295 | try: | ||
227 | 296 | float_is_zero(0.01, precision_digits=3, precision_rounding=0.01) | ||
228 | 297 | except AssertionError: | ||
229 | 298 | pass | ||
230 | 299 | try: | ||
231 | 300 | float_compare(0.01, 0.02, precision_digits=3, precision_rounding=0.01) | ||
232 | 301 | except AssertionError: | ||
233 | 302 | pass | ||
234 | 303 | try: | ||
235 | 304 | float_round(0.01, precision_digits=3, precision_rounding=0.01) | ||
236 | 305 | except AssertionError: | ||
237 | 306 | pass | ||
238 | 147 | 307 | ||
239 | === modified file 'openerp/osv/fields.py' | |||
240 | --- openerp/osv/fields.py 2011-11-28 12:45:35 +0000 | |||
241 | +++ openerp/osv/fields.py 2011-12-21 01:14:24 +0000 | |||
242 | @@ -45,6 +45,7 @@ | |||
243 | 45 | import openerp.netsvc as netsvc | 45 | import openerp.netsvc as netsvc |
244 | 46 | import openerp.tools as tools | 46 | import openerp.tools as tools |
245 | 47 | from openerp.tools.translate import _ | 47 | from openerp.tools.translate import _ |
246 | 48 | from openerp.tools import float_round, float_repr | ||
247 | 48 | 49 | ||
248 | 49 | def _symbol_set(symb): | 50 | def _symbol_set(symb): |
249 | 50 | if symb == None or symb == False: | 51 | if symb == None or symb == False: |
250 | @@ -229,17 +230,20 @@ | |||
251 | 229 | def __init__(self, string='unknown', digits=None, digits_compute=None, required=False, **args): | 230 | def __init__(self, string='unknown', digits=None, digits_compute=None, required=False, **args): |
252 | 230 | _column.__init__(self, string=string, required=required, **args) | 231 | _column.__init__(self, string=string, required=required, **args) |
253 | 231 | self.digits = digits | 232 | self.digits = digits |
254 | 233 | # synopsis: digits_compute(cr) -> (precision, scale) | ||
255 | 232 | self.digits_compute = digits_compute | 234 | self.digits_compute = digits_compute |
256 | 233 | if required: | 235 | if required: |
257 | 234 | warnings.warn("Making a float field `required` has no effect, as NULL values are " | 236 | warnings.warn("Making a float field `required` has no effect, as NULL values are " |
258 | 235 | "automatically turned into 0.0", PendingDeprecationWarning, stacklevel=2) | 237 | "automatically turned into 0.0", PendingDeprecationWarning, stacklevel=2) |
259 | 236 | 238 | ||
260 | 237 | |||
261 | 238 | def digits_change(self, cr): | 239 | def digits_change(self, cr): |
262 | 239 | if self.digits_compute: | 240 | if self.digits_compute: |
266 | 240 | t = self.digits_compute(cr) | 241 | self.digits = self.digits_compute(cr) |
267 | 241 | self._symbol_set=('%s', lambda x: ('%.'+str(t[1])+'f') % (__builtin__.float(x or 0.0),)) | 242 | if self.digits: |
268 | 242 | self.digits = t | 243 | precision, scale = self.digits |
269 | 244 | self._symbol_set = ('%s', lambda x: float_repr(float_round(__builtin__.float(x or 0.0), | ||
270 | 245 | precision_digits=scale), | ||
271 | 246 | precision_digits=scale)) | ||
272 | 243 | 247 | ||
273 | 244 | class date(_column): | 248 | class date(_column): |
274 | 245 | _type = 'date' | 249 | _type = 'date' |
275 | @@ -990,11 +994,14 @@ | |||
276 | 990 | self._symbol_set = integer._symbol_set | 994 | self._symbol_set = integer._symbol_set |
277 | 991 | 995 | ||
278 | 992 | def digits_change(self, cr): | 996 | def digits_change(self, cr): |
284 | 993 | if self.digits_compute: | 997 | if self._type == 'float': |
285 | 994 | t = self.digits_compute(cr) | 998 | if self.digits_compute: |
286 | 995 | self._symbol_set=('%s', lambda x: ('%.'+str(t[1])+'f') % (__builtin__.float(x or 0.0),)) | 999 | self.digits = self.digits_compute(cr) |
287 | 996 | self.digits = t | 1000 | if self.digits: |
288 | 997 | 1001 | precision, scale = self.digits | |
289 | 1002 | self._symbol_set = ('%s', lambda x: float_repr(float_round(__builtin__.float(x or 0.0), | ||
290 | 1003 | precision_digits=scale), | ||
291 | 1004 | precision_digits=scale)) | ||
292 | 998 | 1005 | ||
293 | 999 | def search(self, cr, uid, obj, name, args, context=None): | 1006 | def search(self, cr, uid, obj, name, args, context=None): |
294 | 1000 | if not self._fnct_search: | 1007 | if not self._fnct_search: |
295 | 1001 | 1008 | ||
296 | === modified file 'openerp/tools/__init__.py' | |||
297 | --- openerp/tools/__init__.py 2011-06-23 09:03:57 +0000 | |||
298 | +++ openerp/tools/__init__.py 2011-12-21 01:14:24 +0000 | |||
299 | @@ -31,6 +31,7 @@ | |||
300 | 31 | from pdf_utils import * | 31 | from pdf_utils import * |
301 | 32 | from yaml_import import * | 32 | from yaml_import import * |
302 | 33 | from sql import * | 33 | from sql import * |
303 | 34 | from float_utils import * | ||
304 | 34 | 35 | ||
305 | 35 | #.apidoc title: Tools | 36 | #.apidoc title: Tools |
306 | 36 | 37 | ||
307 | 37 | 38 | ||
308 | === added file 'openerp/tools/float_utils.py' | |||
309 | --- openerp/tools/float_utils.py 1970-01-01 00:00:00 +0000 | |||
310 | +++ openerp/tools/float_utils.py 2011-12-21 01:14:24 +0000 | |||
311 | @@ -0,0 +1,186 @@ | |||
312 | 1 | # -*- coding: utf-8 -*- | ||
313 | 2 | ############################################################################## | ||
314 | 3 | # | ||
315 | 4 | # OpenERP, Open Source Business Applications | ||
316 | 5 | # Copyright (c) 2011 OpenERP S.A. <http://openerp.com> | ||
317 | 6 | # | ||
318 | 7 | # This program is free software: you can redistribute it and/or modify | ||
319 | 8 | # it under the terms of the GNU Affero General Public License as | ||
320 | 9 | # published by the Free Software Foundation, either version 3 of the | ||
321 | 10 | # License, or (at your option) any later version. | ||
322 | 11 | # | ||
323 | 12 | # This program is distributed in the hope that it will be useful, | ||
324 | 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
325 | 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
326 | 15 | # GNU Affero General Public License for more details. | ||
327 | 16 | # | ||
328 | 17 | # You should have received a copy of the GNU Affero General Public License | ||
329 | 18 | # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
330 | 19 | # | ||
331 | 20 | ############################################################################## | ||
332 | 21 | |||
333 | 22 | import math | ||
334 | 23 | |||
335 | 24 | def _float_check_precision(precision_digits=None, precision_rounding=None): | ||
336 | 25 | assert (precision_digits is not None or precision_rounding is not None) and \ | ||
337 | 26 | not (precision_digits and precision_rounding),\ | ||
338 | 27 | "exactly one of precision_digits and precision_rounding must be specified" | ||
339 | 28 | if precision_digits is not None: | ||
340 | 29 | return 10 ** -precision_digits | ||
341 | 30 | return precision_rounding | ||
342 | 31 | |||
343 | 32 | def float_round(value, precision_digits=None, precision_rounding=None): | ||
344 | 33 | """Return ``value`` rounded to ``precision_digits`` | ||
345 | 34 | decimal digits, minimizing IEEE-754 floating point representation | ||
346 | 35 | errors, and applying HALF-UP (away from zero) tie-breaking rule. | ||
347 | 36 | Precision must be given by ``precision_digits`` or ``precision_rounding``, | ||
348 | 37 | not both! | ||
349 | 38 | |||
350 | 39 | :param float value: the value to round | ||
351 | 40 | :param int precision_digits: number of fractional digits to round to. | ||
352 | 41 | :param float precision_rounding: decimal number representing the minimum | ||
353 | 42 | non-zero value at the desired precision (for example, 0.01 for a | ||
354 | 43 | 2-digit precision). | ||
355 | 44 | :return: rounded float | ||
356 | 45 | """ | ||
357 | 46 | rounding_factor = _float_check_precision(precision_digits=precision_digits, | ||
358 | 47 | precision_rounding=precision_rounding) | ||
359 | 48 | if rounding_factor == 0 or value == 0: return 0.0 | ||
360 | 49 | |||
361 | 50 | # NORMALIZE - ROUND - DENORMALIZE | ||
362 | 51 | # In order to easily support rounding to arbitrary 'steps' (e.g. coin values), | ||
363 | 52 | # we normalize the value before rounding it as an integer, and de-normalize | ||
364 | 53 | # after rounding: e.g. float_round(1.3, precision_rounding=.5) == 1.5 | ||
365 | 54 | |||
366 | 55 | # TIE-BREAKING: HALF-UP | ||
367 | 56 | # We want to apply HALF-UP tie-breaking rules, i.e. 0.5 rounds away from 0. | ||
368 | 57 | # Due to IEE754 float/double representation limits, the approximation of the | ||
369 | 58 | # real value may be slightly below the tie limit, resulting in an error of | ||
370 | 59 | # 1 unit in the last place (ulp) after rounding. | ||
371 | 60 | # For example 2.675 == 2.6749999999999998. | ||
372 | 61 | # To correct this, we add a very small epsilon value, scaled to the | ||
373 | 62 | # the order of magnitude of the value, to tip the tie-break in the right | ||
374 | 63 | # direction. | ||
375 | 64 | # Credit: discussion with OpenERP community members on bug 882036 | ||
376 | 65 | |||
377 | 66 | normalized_value = value / rounding_factor # normalize | ||
378 | 67 | epsilon_magnitude = math.log(abs(normalized_value), 2) | ||
379 | 68 | epsilon = 2**(epsilon_magnitude-53) | ||
380 | 69 | normalized_value += cmp(normalized_value,0) * epsilon | ||
381 | 70 | rounded_value = round(normalized_value) # round to integer | ||
382 | 71 | result = rounded_value * rounding_factor # de-normalize | ||
383 | 72 | return result | ||
384 | 73 | |||
385 | 74 | def float_is_zero(value, precision_digits=None, precision_rounding=None): | ||
386 | 75 | """Returns true if ``value`` is small enough to be treated as | ||
387 | 76 | zero at the given precision (smaller than the corresponding *epsilon*). | ||
388 | 77 | The precision (``10**-precision_digits`` or ``precision_rounding``) | ||
389 | 78 | is used as the zero *epsilon*: values less than that are considered | ||
390 | 79 | to be zero. | ||
391 | 80 | Precision must be given by ``precision_digits`` or ``precision_rounding``, | ||
392 | 81 | not both! | ||
393 | 82 | |||
394 | 83 | Warning: ``float_is_zero(value1-value2)`` is not equivalent to | ||
395 | 84 | ``float_compare(value1,value2) == 0``, as the former will round after | ||
396 | 85 | computing the difference, while the latter will round before, giving | ||
397 | 86 | different results for e.g. 0.006 and 0.002 at 2 digits precision. | ||
398 | 87 | |||
399 | 88 | :param int precision_digits: number of fractional digits to round to. | ||
400 | 89 | :param float precision_rounding: decimal number representing the minimum | ||
401 | 90 | non-zero value at the desired precision (for example, 0.01 for a | ||
402 | 91 | 2-digit precision). | ||
403 | 92 | :param float value: value to compare with the precision's zero | ||
404 | 93 | :return: True if ``value`` is considered zero | ||
405 | 94 | """ | ||
406 | 95 | epsilon = _float_check_precision(precision_digits=precision_digits, | ||
407 | 96 | precision_rounding=precision_rounding) | ||
408 | 97 | return abs(float_round(value, precision_rounding=epsilon)) < epsilon | ||
409 | 98 | |||
410 | 99 | def float_compare(value1, value2, precision_digits=None, precision_rounding=None): | ||
411 | 100 | """Compare ``value1`` and ``value2`` after rounding them according to the | ||
412 | 101 | given precision. A value is considered lower/greater than another value | ||
413 | 102 | if their rounded value is different. This is not the same as having a | ||
414 | 103 | non-zero difference! | ||
415 | 104 | Precision must be given by ``precision_digits`` or ``precision_rounding``, | ||
416 | 105 | not both! | ||
417 | 106 | |||
418 | 107 | Example: 1.432 and 1.431 are equal at 2 digits precision, | ||
419 | 108 | so this method would return 0 | ||
420 | 109 | However 0.006 and 0.002 are considered different (this method returns 1) | ||
421 | 110 | because they respectively round to 0.01 and 0.0, even though | ||
422 | 111 | 0.006-0.002 = 0.004 which would be considered zero at 2 digits precision. | ||
423 | 112 | |||
424 | 113 | Warning: ``float_is_zero(value1-value2)`` is not equivalent to | ||
425 | 114 | ``float_compare(value1,value2) == 0``, as the former will round after | ||
426 | 115 | computing the difference, while the latter will round before, giving | ||
427 | 116 | different results for e.g. 0.006 and 0.002 at 2 digits precision. | ||
428 | 117 | |||
429 | 118 | :param int precision_digits: number of fractional digits to round to. | ||
430 | 119 | :param float precision_rounding: decimal number representing the minimum | ||
431 | 120 | non-zero value at the desired precision (for example, 0.01 for a | ||
432 | 121 | 2-digit precision). | ||
433 | 122 | :param float value1: first value to compare | ||
434 | 123 | :param float value2: second value to compare | ||
435 | 124 | :return: (resp.) -1, 0 or 1, if ``value1`` is (resp.) lower than, | ||
436 | 125 | equal to, or greater than ``value2``, at the given precision. | ||
437 | 126 | """ | ||
438 | 127 | rounding_factor = _float_check_precision(precision_digits=precision_digits, | ||
439 | 128 | precision_rounding=precision_rounding) | ||
440 | 129 | value1 = float_round(value1, precision_rounding=rounding_factor) | ||
441 | 130 | value2 = float_round(value2, precision_rounding=rounding_factor) | ||
442 | 131 | delta = value1 - value2 | ||
443 | 132 | if float_is_zero(delta, precision_rounding=rounding_factor): return 0 | ||
444 | 133 | return -1 if delta < 0.0 else 1 | ||
445 | 134 | |||
446 | 135 | def float_repr(value, precision_digits): | ||
447 | 136 | """Returns a string representation of a float with the | ||
448 | 137 | the given number of fractional digits. This should not be | ||
449 | 138 | used to perform a rounding operation (this is done via | ||
450 | 139 | :meth:`~.float_round`), but only to produce a suitable | ||
451 | 140 | string representation for a float. | ||
452 | 141 | |||
453 | 142 | :param int precision_digits: number of fractional digits to | ||
454 | 143 | include in the output | ||
455 | 144 | """ | ||
456 | 145 | # Can't use str() here because it seems to have an intrisic | ||
457 | 146 | # rounding to 12 significant digits, which causes a loss of | ||
458 | 147 | # precision. e.g. str(123456789.1234) == str(123456789.123)!! | ||
459 | 148 | return ("%%.%sf" % precision_digits) % value | ||
460 | 149 | |||
461 | 150 | |||
462 | 151 | if __name__ == "__main__": | ||
463 | 152 | |||
464 | 153 | import time | ||
465 | 154 | start = time.time() | ||
466 | 155 | count = 0 | ||
467 | 156 | errors = 0 | ||
468 | 157 | |||
469 | 158 | def try_round(amount, expected, precision_digits=3): | ||
470 | 159 | global count, errors; count += 1 | ||
471 | 160 | result = float_repr(float_round(amount, precision_digits=precision_digits), | ||
472 | 161 | precision_digits=precision_digits) | ||
473 | 162 | if result != expected: | ||
474 | 163 | errors += 1 | ||
475 | 164 | print '###!!! Rounding error: got %s , expected %s' % (result, expected) | ||
476 | 165 | |||
477 | 166 | # Extended float range test, inspired by Cloves Almeida's test on bug #882036. | ||
478 | 167 | fractions = [.0, .015, .01499, .675, .67499, .4555, .4555, .45555] | ||
479 | 168 | expecteds = ['.00', '.02', '.01', '.68', '.67', '.46', '.456', '.4556'] | ||
480 | 169 | precisions = [2, 2, 2, 2, 2, 2, 3, 4] | ||
481 | 170 | for magnitude in range(7): | ||
482 | 171 | for i in xrange(len(fractions)): | ||
483 | 172 | frac, exp, prec = fractions[i], expecteds[i], precisions[i] | ||
484 | 173 | for sign in [-1,1]: | ||
485 | 174 | for x in xrange(0,10000,97): | ||
486 | 175 | n = x * 10**magnitude | ||
487 | 176 | f = sign * (n + frac) | ||
488 | 177 | f_exp = ('-' if f != 0 and sign == -1 else '') + str(n) + exp | ||
489 | 178 | try_round(f, f_exp, precision_digits=prec) | ||
490 | 179 | |||
491 | 180 | stop = time.time() | ||
492 | 181 | |||
493 | 182 | # Micro-bench results: | ||
494 | 183 | # 47130 round calls in 0.422306060791 secs, with Python 2.6.7 on Core i3 x64 | ||
495 | 184 | # with decimal: | ||
496 | 185 | # 47130 round calls in 6.612248100021 secs, with Python 2.6.7 on Core i3 x64 | ||
497 | 186 | print count, " round calls, ", errors, "errors, done in ", (stop-start), 'secs' | ||
498 | 0 | 187 | ||
499 | === modified file 'openerp/tools/misc.py' | |||
500 | --- openerp/tools/misc.py 2011-09-22 09:54:43 +0000 | |||
501 | +++ openerp/tools/misc.py 2011-12-21 01:14:24 +0000 | |||
502 | @@ -1200,4 +1200,4 @@ | |||
503 | 1200 | def __missing__(self, key): | 1200 | def __missing__(self, key): |
504 | 1201 | return unquote(key) | 1201 | return unquote(key) |
505 | 1202 | 1202 | ||
507 | 1203 | # vim:expandtab:smartindent:tabstop=4:softtabstop=4:shiftwidth=4: | 1203 | # vim:expandtab:smartindent:tabstop=4:softtabstop=4:shiftwidth=4: |
508 | 1204 | \ No newline at end of file | 1204 | \ No newline at end of file |
I hope this rounding methods will be used to replace the problematic checks
not an easy task to find all occurencies