* More Financial function extracts, this time looking at the Periodic Cashflow functions
* Initial extract of Constant Periodic Interest and Payment functions
* Continue MathTrig Breakup - Completion!
Continuing the process of breaking MathTrip.php up into smaller classes. This round takes care of everything that was left:
- ABS
- DEGREES
- EXP
- RADIANS
- SQRT
- SQRTPI
- SUMSQ, SUMX2MY2, SUMX2PY2, SUMXMY2
The only notable logic change was that the 3 SUMX* functions had accepted arrays of unlike length; in that condition, they now return N/A, as Excel does. There had been no tests for this condition.
All the functions in MathTrig.php are now deprecated. Except for COMBIN, the test suite executes them only from MathTrig MovedFunctionsTest. COMBIN is still directly called by some Statistics Binomial functions which have not yet had the opportunity to be re-coded for the new location.
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
* Let's start with some appeasements to phpstan, just to reduce the baseline
* Appeasements to phpstan, taking the number of reported errors down to just 61
* Extract Normal and Standard Normal Distributions from the Statistical Class
* Extract ZTest from the Statistical Class, and move it to the Standard Normal Distribution class
Additional unit tests for NORMINV()
* Extract LogNormal distribution functions from Statistical
* Continue MathTrig Breakup - Penultimate?
Continuing the process of breaking MathTrip.php up into smaller classes. This round takes care of about half of what is left, so perhaps one round after this one will finish the job:
- ARABIC
- COMBIN; also implemented COMBINA
- FACTDOUBLE
- GCD (which accepts and ignores empty cells as arguments, but returns VALUE if all the arguments are that way; LCM does the same)
- LOG_BASE, LOG10, LN
- implemented MUNIT
- MOD
- POWER
- RAND, RANDBETWEEN (RANDARRAY is too complicated to implement with this ticket)
As you can see from the description, there are some functions which were combined in a single class. When not combined, I adopted PowerKiki's suggestion of using "execute" as the function name.
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
* Extract a few more Distribution functions from Statistical; this time EXPONDIST() and HYPGEOMDIST()
* Extract the F Distribution (although only F.DIST() is implemented so far
* Updae docblocks
* PHPCS
* Extract Binomial Distribution functions from Statistical
Replace the old MS algorithm for CRITBINOM() (which has now been replaced with te BINOM.INV() function) with a brute force approach - I'll look to refine it later. The MS algorithm is no longer documented, and the implementation produced erroneous results anyway
* Exract the NEGBINOMDIST() function as well; still need to add a cumulative flag to support the additional argument for the newer NEGBINOM.DIST() function
* Rationalise validation of probability arguments
* Difference in variance calculations between Excel/Gnumeric and Open/LibreOffice
* Simplify STDEV() function logic by remembering that STDEV() is simply the square root of VAR(), so we can simply use the VAR() calculaion rather than duplicating the basic logic... and also allow for the differences between Excel/Gnumeric and Open/LibreOffice
* Start implementing Newton-Raphson for the inverse of Statistical Distributions, starting with the two-tailed Student-T
* Additional unit tests and validations
* Use the new Newton Raphson class for calculating the Inverse of ChiSquared
* Extract Weibull distribution, and provide unit tests
* Extract ACCRINT() and ACCRINTM() Financial functions into their own class
Implement additional validations, with additional unit tests
Add support for the new calculation method argument for ACCRINT()
* Additional tests for Amortization functions
Continuing the process of breaking MathTrip.php up into smaller classes. This round takes care of all functions which might be an impediment to installing due to either uncovered code or "complexity":
- BASE
- FACT
- LCM
- MDETERM, MINVERSE, MMULT
- MULTINOMIAL
- PRODUCT
- QUOTIENT
- SERIESSUM
- SUM
- SUMPRODUCT
MathTrig and the members in directory MathTrig are now 100% covered. Many tests have been added, and some edge-case bugs are corrected. Some cases where PhpSpreadsheet had rejected numeric values stored as strings have been changed to accept them whenever Excel does; there had been no tests for that condition.
Boolean arguments are now accepted as arguments wherever Excel accpets them. Taking a cue from what has been done in Engineering, the parameter validation now happens in a routine which issues Exceptions for invalid values; this simplifies the code in the functions themselves. Thank you for doing that; I did not foresee how useful that was when I first looked at it.
Consistent with earlier changes of this nature, the versions in the MathTrig class remain, with a doc block indicating deprecation, and a stub call to the new routines.
All tests except for MINVERSE and MMULT are now handled in the context of a spreadsheet rather than a direct call to the calculation function which implements it. PhpSpreadsheet would need to handle dynamic arrays in order to test MINVERSE and MMULT in a spreadsheet context. Implementing that looks like it might be *very* challenging. It is not something I plan to look at, at least not in the near future.
One parsing problem turned up in the test conversion. It is in one of the SUMIF tests. It takes me to an area in Calculation where the comment says "I don't even want to know what you did to get here". It did not show up in the previous incarnation because, by using a direct call, the previous test managed to bypass the parsing. I have confirmed that this problem shows up in earlier releases of PhpSpreadsheet, so the changes in this PR did not cause it - they merely exposed it. I have left the test intact, but marked it "incomplete" for documentation purposes. I have not been able to get a handle on what's going wrong yet. I will probably open an issue on it if I can't resolve it soon. However, the test in question isn't a "real world" issue, and the error wasn't caused by this change, so I see no reason to delay this pending a resolution of the problem.
SUM had an idiosyncratic moment of its own. It had been ignoring non-numeric values, but Excel returns VALUE in that situation. So I changed it and wrote some new tests, which worked, but ... SUMIF uses several levels of indirection to get to SUM, and SUMIF *does* ignore non-numeric values, so a SUMIF test broke. SUM is a really simple function; the most practical approach seemed to be to clone it, with the string-accepting version being used by the Legacy version (which is called by SUMIF), and the non-string-accepting version being used in the Calculation Function table. That seems far easier and more practical than, for instance, adding a boolean parameter to the variable parameter list. As a follow-up, I will change SUMIF to explicitly call the appropriate new version, but I did not want to add that to this already large change.
SUM again - although it was fully covered beforehand, there was not a specific test member for it. There is now.
FACT had been coded to fail Gnumeric requests where the numeric argument has a decimal portion. However, Gnumeric does accept such an argument, and, unlike Excel and ODS, does not truncate it, but returns the result of a Gamma function call instead. This has been corrected.
When LCM included arguments which contained both 0 and a negative number, it returned 0 or NUM, whichever it found first. It is changed to always return NUM in that circumstance, as Excel does.
QUOTIENT had been documented as taking a variadic list of arguments. In fact, it takes exactly 2 - numerator and denominator - and the docblock and signature is fixed, even in the deprecated version.
The SERIESSUM docbock and signature are more accurate, even in the deprecated version. It is changed to ignore nulls, as Excel does, rather than return VALUE, and is one of the routines which previously rejected numbers in string form.
SUBTOTAL tests had used mocking for some reason. These are replaced with normal tests. And SUBTOTAL had a big surprise in store. That part of it which deals with hidden cells cares only whether the row is hidden, and doesn't care about the column's visibility.
I struggled with whether it should be SubTotal or Subtotal. I think the latter is correct, so that's how I proceeded. I don't think there are likely to be any other capitalization controversies.
* New Bessel Algorithm, providing a higher degree of precision (12 decimal places) and still matching/exceeding MS Excel's precision across the range of values
* Complete Breakup Of Calculation/DateTime Functions
In conjunction with parallel breakups happening in other areas of Calculation, this change breaks up all the DateTime functions into their own classes. All methods remaining in DateTime itself have a doc block deprecation notice, and consist only of stub code to call the replacement methods. Coverage of DateTime itself and all the replacement methods is 100%.
There is only one substantive change to the code (see next paragraph). Among the non-substantive changes, it now adopts the same parsing technique (throwing and catching exceptions) already in use in Engineering and MathTrig. Boolean parameters are allowed in lieu of numbers when Excel allows them. Most of the code changes involve refactoring due to the need to avoid Scrutinizer "complexity" failures in what it will consider to be new methods.
Issue #1936 was opened just as I was staging this. It is now fixed. One existing WORKDAY test was wrong (noted in a comment in the test data file), and a bunch of new tests are added.
I found it confusing to use DateTime as a node of the the class name since most of the methods invoke native DateTime methods. So, everything is moved to directory DateTimeExcel, and that is what is used in the class names.
There are several follow-up activities that I am planning to undertake if this PR is merged.
- ODS supports dates well before 1900. There are exactly 2 assertions for this functionality. More are needed (and some functions might have to change to accept this).
- WEEKDAY has some poorly documented extra options for "style" which are not yet implemented.
- Most tests have been changed to use a formula as entered on a spreadsheet rather than a direct call to the method which implements the formula. There are 3 exceptions at this time. WORKDAY and NETWORKDAYS, which include arrays as part of their parameters, are more complicated than most. YEARFRAC was just too large to deal with now.
- There are direct calls to the now-deprecated methods in both source code and tests, mostly in Financial code, but possibly in others as well. These need to be changed.
- Some constants, none "officially" documented, remain in the original class. These should be either deleted or marked deprecated. I wasn't sure if deprecation was even possible (or desirable), and did not want that to be something which would cause Scrutinizer to fail the change.
* Deprecate Now-unused Constants, Fix Yearfrac bug, Change 3 Tests
Add new DateTime/Constants class, initially populated with constants used in Weeknum.
MS has another inconsistency with how it handles null cells in Yearfrac. Change PhpSpreadsheet to behave compatibly with this bug.
I have modified YearFrac, WorkDay, and NetworkDays tests to be more to my liking. Many tests added to YearFrac because of the bug above. Only minor modifications to the existing tests for the others.
* Start work on breaking down some of the Financial Excel functions
* Unhappy path unit tests for Treasury Bill functions
* Codebase for Treasury Bills includes logic for a different days between settlement and maturity calculation for OpenOffice; but Open/Libre Office now uses the Excel days calculation, so this discrepancy between packages is no longer required
* We've already converted the Settlement and Maturity dates to Excel timestamps, so there's no need to try doing it again when calculating the days between Settlement and Maturity
* Add Unit Tests for the Days per Year helper function
* Extract Interest Rate functions - EFFECT() and NOMINAL() - with additional validation, and unhappy path unit tests
* First pass at extracting the Coupon Excel functions
* Simplify the validation methods
* Extended unit tests to cover all combinations of frequency and basis, including leap years
Fix for COUPDAYSNC() when basis is US 360 and settlement date is the last day of the month
* Ensure that all Financial function code uses the new Helpers class for Days Per Year
* jpgraph seems to be finally dying with PHP. Until we have a valid alternative, disabling this run for PHP because it errors
https://github.com/HuasoFoundries/jpgraph looks like a natural successor, but it isn't BC so it will require some work to integrate
* Bitwise Functions and 32-bit
When running the test suite with 32-bit PHP, a failure was reported in BITLSHIFT.
In fact, all of the following are vulnerable to problems, and didn't report
any failures only because of a scarcity of tests:
- BITAND
- BITOR
- BITXOR
- BITRSHIFT
- BITLSHIFT
Those last 2 can be resolved fairly easily by using multiplication by a power of 2
rather than shifting. The first 3 are a tougher nut to crack, and I will continue
to think how they might best be approached. For now, I have added skippable tests
for each of them, which at least documents the problem.
Aside from adding many new tests, some bugs were correctd:
- The function list in Calculation.php pointed BITXOR to BITOR.
- All 5 functions allow null/false/true parameters.
- BIT*SHIFT shift amount must be numeric, can be negative, allows decimal portion
(which is truncated to integer), and has an absolute value limit of 53.
- Because BITRSHIFT allows negative shift amount, its result can overflow
(in which case return NAN).
- All 5 functions disallow negative parameters (except ...SHIFT second parameter).
This was coded, but the code had been thwarted by an earlier is_int test.
* Full Support for AND/OR/XOR on 32-bit
Previous version did not support operands 2**32 through 2**48.
* Improve Coverage of BIN2DEC etc.
The following functions have some special handling
depending on the Calculation mode:
- BIN2DEC
- BIN2HEX
- BIN2OCT
- DEC2BIN
- DEC2HEX
- DEC2OCT
- HEX2BIN
- HEX2DEC
- HEX2OCT
- OCT2BIN
- OCT2DEC
- OCT2HEX
Ods accepts boolean for its numeric argument.
This had already been coded, but there were no tests for it.
Gnumeric allows the use of non-integer argument where Excel/Ods do not.
The existing code allowed this for certain functions but not for others.
Gnumeric consistently allows it, so there is no need for parameter
gnumericCheck in convertBase::ValidateValue.
Again, there were no tests for this.
There were some minor changes needed:
- In functions where you are allowed to specify the numnber of "places" in the
result, there is an upper bound of 10 which had not been enforced.
- Negative values were not handled correctly in some cases.
- There was at least one (avoidable) error on a 32-bit system.
- Some upper and lower bounds were not being enforced. In addition to enforcing
those, the bounds are now defined as class constants in ConvertDecimal.
Many tests have been added, so that Engineering is now almost 100% covered.
The exception is some BESSEL code. There have been some recent changes to
BESSEL which are not yet part of my fork, so I could not address those now.
However, I freely admit that, when I looked at the uncovered portion, it seemed
like it might be a difficult task, so I probably wouldn't have tackled it anyhow.
In particular, the uncovered code seemed to deal with very large numbers,
and, although PhpSpreadsheet and Excel both give very large results for these
conditions, their answers are not particularly close to each other. I think
we're dealing with resuts approaching infinity. More study is needed.
* Coverage for Helper/Samples
I was perplexed by the fact that Helper/Samples seemed to be entirely uncovered when running the test suite, since I know all the samples are run as part of the test. I think that what must be happening is that the Helper code is invoked mostly as part of a Data Provider (and therefore not counted), not as part of the test proper (which would count). So, this change adds a small number of tests which result in Samples being 100% covered.
Covering one statement was tricky - simulating the inability to create a test directory. Mocking, a technique I have not used before, solves this problem admirably.
* Suggestions From Mark Baker
Tests changed from assertEquals to assertSame.
Added @covers annotation to test class.
Validate parameter for method being mocked.
* First step extracting INDIRECT() and OFFSET() to their own classes
* Start building unit tests for OFFSET() and INDEX()
* Named ranges should be handled by the Calculation Engine, not by the implementation of the Excel INDIRECT() function
* When calling the calculation engine to get the range of cells to return, INDIRECT() and OFFSET() should use the instance of the calculation engine for the current workbook to benefit from cached results in that range
There's a couple of minor bugfixes in here; but it's basically just refactoring of the INDIRECT() and OFFSET() Excel functions into their own classes - still needs a lot of work on unit testing; and there's a lot more that could be improved in the code itself (including handling of the a1 flag for R1C1 format in INDIRECT()
* Continue MathTrig Breakup - Trig Functions
Continuing the process of breaking MathTrip.php up into smaller classes.
This round takes care of the trig and hyperbolic functions, plus a few others.
- COS, COSH, ACOS, ACOSH
- COT, COTH, ACOT, ACOTH
- CSC, CSCH
- SEC, SECH
- SIN, SINH, ASIN, ASINH
- TAN, TANH, ATAN, ATANH, ATAN2
- EVEN
- ODD
- SIGN
There are no bug fixes in this PR, except that boolean arguments are now
accepted for all these functions, as they are for Excel.
Taking a cue from what has been done in Engineering, the parameter validation
now happens in a routine which issues Exceptions for invalid values;
this simplifies the code in the functions themselves.
Consistent with earlier changes of this nature, the versions in the
MathTrig class remain, with a doc block indicating deprecation,
and a stub call to the new routines.
I think several more iterations will be needed to break up MathTrig completely.
* Extract LookupRef\INDEX() into index() method of LookupRef\Matrix class
Additional tests
* Bugfix for returning a column using INDEX()
* Some improvements to ROW() and COLUMN()
* Simplify some of the INDEX() logic, eliminating redundant code
Adrien Crivelli
Akira Taniguchi
Mark Baker
Vivek
oleibman