Rounding up or truncation?

[Matty F]

I have an electronic digital calliper that shows four decimal places of an inch for imperial measurements (and almost everything around here is still Imperial and will stay that way because they are 100 years old). I actually want to see fractions of inches, e.g. I want to see 1/32" instead of 0.03125", but that appears to not be possible in an electronic caliper. A vernier calliper does show fractions very well but some people can't read verniers, and electronic is quicker.

So I made a chart using Excel to show all of the 64ths of an inch as 4 decimal digits. That shows 1/32" as 0.0313" because Excel has rounded up from 0.03125", correctly I believe. The British and US engineering tables show 1/32" as 0.0312", so they have truncated instead of rounding. And many other figures are truncated as well. I realise that the measurements will be less accurate than 0.00005" but that's not the point. My chart is different from the official charts that were probably calculated with a slide rule or an abacus

100 years ago. Isn't it accepted these days to round upwards and not truncate? My chart is printed nice and big so I can read it without a magnifying glass. And it's on one laminated A4 page instead of 5 pages in a tiny book. So far 4 people agree that it should be rounded, and 2 don't agree because they trust what is in a book.

So, round or truncate? I am not talking about prices in a shop, but measurements.

[Alan Braggins]

There's no one generally accepted rule. Can you tell from other numbers in the chart whether they are rounding down, or rounding to even? The latter avoids systematic bias, and is fairly common (I expect Excel has an option for it somewhere if you want to make your chart match).

[RobertL]

Take care using Excel for drawing up tables at high precision:

[RobertL]

Another thing to bear in mind with old sets of tables is that the inch was redfined in 1959 to be exactly 25.4mm.

Tables drawn up before 1959 might use a different sized inch.

Robert

[Matty F]

match).

top Google result for imperial fraction conversion rounds the way that I say is correct, i.e. 0 1 2 3 4 rounds down and 5 6 7 8 9 rounds up. Where is the systematic bias in that?

[polygonum]

up.

As I see it, it is the issue of always rounding the mid-point value (i.e.

5) in a specific direction that is a cause of bias. Hence the other approaches that attempt to balance that out. Effectively the sum of all values rises due to your always rounding up 5.

[Tim Streater]

This is correct rounding. But you could display as 5 decimal digits. Or you could add a formula in the cell to control the number of displayed digits based on the 0.00005 part.

[Tim Streater]

numbers. AIUI, Matty is just going to display them. And surely also Excel has the "true" value [1] internally anyway, so ISTM that if you did add them in Excel, you'd get an unbiassed result.

[1] Within the limits of floating point precision, anyway.

[Scott M]

Depends what you're going to do with the result. If machining two components, say, and you know whether the fit should be tight or loose you use this to adjust your numbers. If these components are going to be a shrink or push fit then you'd make the inner one larger and the outer one smaller. If there's to be a sliding fit then the reverse would apply.

[Matty F]

But the tables in the engineering handbooks appear to truncate, which is far less accurate.

[Matty F]

The callipers display 4 decimal places so I want only 4 in my table. In fact the 4th digit measured will be very inaccurate as we are mainly measuring used twist drills that are possibly damaged, and looking up a similar reading in my table and saying, yes that's a

27/64th drill. I have made a gauge that shows the sizes almost instantly, but sometimes a metric drill gets put in with the imperial drills and is close to an imperial size. I've put yellow paint on all the metric drills but even then they get mixed up!

[Phil L]

A farmer told me that he had 68 sheep and asked would I like to round them up, I said, 'OK you've got 70'

[Scott M]

While I think of it, since you're talking about drill bits, the chances of a drill cutting a hole accurately to a thou is very low so it's not worth worrying about whether the 4th DP should be rounded up or down.

(Oh, and FWIW, I concur with the rounding up for values over 5 being the correct method.)

[polygonum]

You always have to remember that Excel is for bean-counters so total sums are important to them.

But if you are measuring engineering things, you might as well round down. So long as there is sufficient precision for the purpose, whatever that may be, things are often slightly smaller due to wear.

And you can control the rounding pretty much to your heart's content if you put the effort in to using the appropriate functions, etc. If you care enough, then never rely on the display precision mechanism - actually choose what you want done.

[Alan Braggins]

Yes, but if he wants to display them matching the official charts, he'll have to display them using the same rounding method that the official charts do. He already realises that he's displaying them to a precision that he can't measure to and therefore that it doesn't really matter.

[Roger Chapman]

That is the convention I was taught mumble mumble years ago but approximating by rounding down is no less accurate than rounding up.

snip

There can be a difference between truncation and rounding down. The example given above could equally be rounded down rather than truncated. So how would the engineering tables you refer to treat 3/64ths - 0.046875?

[dennis]

Any errors caused by rounding should be covered by the tolerance being worked to. So there will be a range of over and undersized values that will be 1/32. I don't see why there should be an issue, all that is needed is a table that lists the fraction and the allowable range. That range may need to be different for pins and for holes to ensure they fit.

[Andy Champ]

Four digits, 0, 1, 2, 3 and 4.

'nuff said!

Andy

[polygonum]

The zero doesn't need to be rounded, does it?

[Clive George]

Weird innit. But he's right. Look at the sum of 0.1-10.0 in steps of 0.1

- 505. Now round to integer, always rounding the 0.5 up - ans = 510 - it's a bit high. Now try banker's rounding, ie if it's x.5, round to the even number. 0.5 -> 0, 1.5 -> 2, etc. Sum = 505, ie no systematic bias.