These changes have already been implemented twice, and been regressed twice. I'll try once more (with a different approach), then give up ...
As configured, Phpstan running under Php7 reports no errors. However, running under Php8, it reports 100 (!) errors. The vast majority of these are due to two reasons:
- renaming parameters in Php builtin functions in preparation for named parameters.
- using the new class GdImage rather than type resource as the argument type for many image-based functions.
Regardless of the cause, this will be a problem sooner or later. This PR is an attempt to get ahead of that problem. For source members, it mostly adds annotations or updates doc-blocks. Only 2 members have changes to executable code, and these are very minor - BitWise and Writer/Xlsx. For test members, all baseline errors are deleted and the code is fixed. Php7 and Php8 both report no errors with this configuration.
* Implementation of the SEQUENCE() Excel365 function
Note that the Calculation Engine does not yet support the Spill operator, or spilling functions
* Handle the use-case of step = 0; and tests for exception handling for invalid arguments
* Update Change Log
* Handle a wildcard match that contains a forward slash in the pattern by adding / to the delimiter list of preg_quote
* Fix SUMIF doing a wildcard match on empty cells (NULL)
* Fix compare logic to return false when value is an empty string or NULL (Verified against LibreOffice SUMIF and MATCH handling of empty cells)
* Improve Identification of Samples in Coverage Report
The Phpunit coverage report currently contains bullet items like `PhpOffice\PhpSpreadsheetTests\Helper\SampleTest\testSample with data set "49"`. This extremely simple change takes advantage of Phpunit's ability to accept an array with keys which are either strings or integers, by using the sample filenames as the array keys rather than sequential but otherwise meaningless integers (e.g. `49` in the earlier cited item). The bullet item will now read `PhpOffice\PhpSpreadsheetTests\Helper\SampleTest\testSample with data set "Basic/38_Clone_worksheet.php"`.
* Fix for Issue 2158 (AverageIf Calculation Problem)
Issue #2158 reports an error calculating AverageIf because a function returns null rather than a string. There turn out to be several components to this problem:
- The nominal fix to the problem is to add some null-to-nullstring coercion in DatabaseAbstract.
- This fixes the error, but does not necessarily lead to the correct result because buildQuery treats values of null and null-string identically, whereas Excel does not. So change that to treat null-string as any other string.
- But that doesn't lead to the correct result either. That's because Functions/ifCondition recognizes a null string, but then continues to (over-)process it until it returns the wrong result. Fix this problem in conjunction with the other two, and we finally get the correct result.
A new unit test is added for AVERAGEIF, and new test cases are added for SUMIF. In each case, there are complementary tests for conditions of null and null-string, and the results agree with Excel. There may or may not be value in adding new tests to other functions, and I will be glad to do so for any functions which you care to identify, but no existing tests broke as a result of these changes.
As issue #2042 documents, SUM behaves differently with invalid strings depending on whether they come from a cell or are used as literals in the formula. SUM is not alone in this regard; COUNTA is another function within this behavior, and the solution to this one is modeled on COUNTA. New tests are added for SUM, and the resulting tests are duplicated to confirm correct behavior for both cells and literals.
Samples 16 (CSV), 17 (Html), and 21 (PDF) were adversely affected by this problem. 17 and 21 were immediately fixed, but 16 had another problem - Excel was not interpreting the UTF8 currency symbols correctly, even though the file was saved with a BOM. After some experimenting, it appears that the `sep=;` line generated by setExcelCompatibility(true) causes Excel to mis-handle the file. This seems like a bug - there is apparently no way to save a UTF-8 CSV with non-ASCII characters which specifies a non-standard separator which Excel will open correctly. I don't know if this is a recent change or if it is just the case that nobody noticed this problem till now. So, I changed Sample 16 to use setUseBom rather than setExcelCompatibility, which solved its problem. I then added new tests for setExcelCompatibility, with documentation of this problem.
* Continue MathTrig Breakup - Completion!
Continuing the process of breaking MathTrip.php up into smaller classes. This round takes care of everything that was left:
- ABS
- DEGREES
- EXP
- RADIANS
- SQRT
- SQRTPI
- SUMSQ, SUMX2MY2, SUMX2PY2, SUMXMY2
The only notable logic change was that the 3 SUMX* functions had accepted arrays of unlike length; in that condition, they now return N/A, as Excel does. There had been no tests for this condition.
All the functions in MathTrig.php are now deprecated. Except for COMBIN, the test suite executes them only from MathTrig MovedFunctionsTest. COMBIN is still directly called by some Statistics Binomial functions which have not yet had the opportunity to be re-coded for the new location.
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
* Continue MathTrig Breakup - Penultimate?
Continuing the process of breaking MathTrip.php up into smaller classes. This round takes care of about half of what is left, so perhaps one round after this one will finish the job:
- ARABIC
- COMBIN; also implemented COMBINA
- FACTDOUBLE
- GCD (which accepts and ignores empty cells as arguments, but returns VALUE if all the arguments are that way; LCM does the same)
- LOG_BASE, LOG10, LN
- implemented MUNIT
- MOD
- POWER
- RAND, RANDBETWEEN (RANDARRAY is too complicated to implement with this ticket)
As you can see from the description, there are some functions which were combined in a single class. When not combined, I adopted PowerKiki's suggestion of using "execute" as the function name.
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
Continuing the process of breaking MathTrip.php up into smaller classes. This round takes care of all functions which might be an impediment to installing due to either uncovered code or "complexity":
- BASE
- FACT
- LCM
- MDETERM, MINVERSE, MMULT
- MULTINOMIAL
- PRODUCT
- QUOTIENT
- SERIESSUM
- SUM
- SUMPRODUCT
MathTrig and the members in directory MathTrig are now 100% covered. Many tests have been added, and some edge-case bugs are corrected. Some cases where PhpSpreadsheet had rejected numeric values stored as strings have been changed to accept them whenever Excel does; there had been no tests for that condition.
Boolean arguments are now accepted as arguments wherever Excel accpets them. Taking a cue from what has been done in Engineering, the parameter validation now happens in a routine which issues Exceptions for invalid values; this simplifies the code in the functions themselves. Thank you for doing that; I did not foresee how useful that was when I first looked at it.
Consistent with earlier changes of this nature, the versions in the MathTrig class remain, with a doc block indicating deprecation, and a stub call to the new routines.
All tests except for MINVERSE and MMULT are now handled in the context of a spreadsheet rather than a direct call to the calculation function which implements it. PhpSpreadsheet would need to handle dynamic arrays in order to test MINVERSE and MMULT in a spreadsheet context. Implementing that looks like it might be *very* challenging. It is not something I plan to look at, at least not in the near future.
One parsing problem turned up in the test conversion. It is in one of the SUMIF tests. It takes me to an area in Calculation where the comment says "I don't even want to know what you did to get here". It did not show up in the previous incarnation because, by using a direct call, the previous test managed to bypass the parsing. I have confirmed that this problem shows up in earlier releases of PhpSpreadsheet, so the changes in this PR did not cause it - they merely exposed it. I have left the test intact, but marked it "incomplete" for documentation purposes. I have not been able to get a handle on what's going wrong yet. I will probably open an issue on it if I can't resolve it soon. However, the test in question isn't a "real world" issue, and the error wasn't caused by this change, so I see no reason to delay this pending a resolution of the problem.
SUM had an idiosyncratic moment of its own. It had been ignoring non-numeric values, but Excel returns VALUE in that situation. So I changed it and wrote some new tests, which worked, but ... SUMIF uses several levels of indirection to get to SUM, and SUMIF *does* ignore non-numeric values, so a SUMIF test broke. SUM is a really simple function; the most practical approach seemed to be to clone it, with the string-accepting version being used by the Legacy version (which is called by SUMIF), and the non-string-accepting version being used in the Calculation Function table. That seems far easier and more practical than, for instance, adding a boolean parameter to the variable parameter list. As a follow-up, I will change SUMIF to explicitly call the appropriate new version, but I did not want to add that to this already large change.
SUM again - although it was fully covered beforehand, there was not a specific test member for it. There is now.
FACT had been coded to fail Gnumeric requests where the numeric argument has a decimal portion. However, Gnumeric does accept such an argument, and, unlike Excel and ODS, does not truncate it, but returns the result of a Gamma function call instead. This has been corrected.
When LCM included arguments which contained both 0 and a negative number, it returned 0 or NUM, whichever it found first. It is changed to always return NUM in that circumstance, as Excel does.
QUOTIENT had been documented as taking a variadic list of arguments. In fact, it takes exactly 2 - numerator and denominator - and the docblock and signature is fixed, even in the deprecated version.
The SERIESSUM docbock and signature are more accurate, even in the deprecated version. It is changed to ignore nulls, as Excel does, rather than return VALUE, and is one of the routines which previously rejected numbers in string form.
SUBTOTAL tests had used mocking for some reason. These are replaced with normal tests. And SUBTOTAL had a big surprise in store. That part of it which deals with hidden cells cares only whether the row is hidden, and doesn't care about the column's visibility.
I struggled with whether it should be SubTotal or Subtotal. I think the latter is correct, so that's how I proceeded. I don't think there are likely to be any other capitalization controversies.
* Continue MathTrig Breakup - Trig Functions
Continuing the process of breaking MathTrip.php up into smaller classes.
This round takes care of the trig and hyperbolic functions, plus a few others.
- COS, COSH, ACOS, ACOSH
- COT, COTH, ACOT, ACOTH
- CSC, CSCH
- SEC, SECH
- SIN, SINH, ASIN, ASINH
- TAN, TANH, ATAN, ATANH, ATAN2
- EVEN
- ODD
- SIGN
There are no bug fixes in this PR, except that boolean arguments are now
accepted for all these functions, as they are for Excel.
Taking a cue from what has been done in Engineering, the parameter validation
now happens in a routine which issues Exceptions for invalid values;
this simplifies the code in the functions themselves.
Consistent with earlier changes of this nature, the versions in the
MathTrig class remain, with a doc block indicating deprecation,
and a stub call to the new routines.
I think several more iterations will be needed to break up MathTrig completely.
* Support 'Forms' for ROMAN Function
This seems like an exceptionally silly thing for MS to have implemented
(Wikipedia on Roman Numerals: "There is no indication this is anything
other than an invention by the programmer").
Nevertheless, we can, and therefore probably should, implement it.
Not that I can implement it by an algorithm - Excel describes the various extra
styles as "more concise", "more concise", "more concise", and "simplified".
Nevertheless, since the universe of potential calls is relatively small,
it can be implemented as a table of values where the new forms would return
a different value than "classic". This table is relatively large, so I have
put it its own member to avoid overhead when the function is needed.
* Move ROMAN To Its Own Class
See discussion in PR #1837
* PHP 8.1 Deprecations
PHP8.1 Unit tests failed. 1 line fixes are available for
- Shared/Font
- Shared/XMLWriter
- Style/Color
- Writer/HTML
The problem is that an error is also reported for a strcmp at
line 272 of Cell/Cell. Not only does that line not invoke strcmp,
there is no strcmp in all of Cell/Cell, so I don't know what to make
of the error message. Oh well, let's fix what can be fixed.
Still dealing with the mysterious PHP8.1 unit test failure in Cell\Cell,
which seems to have something to do with strcmp. The only uses of
strcmp that I can find in src/ are in Calculation. I can't find any
use of it in test/ or samples/. So, if this doesn't fix the problem,
I may have to give up.
* ROUND Accepts null, false, and true as First Parameter
Issue #1789 was addressed by PR #1799. In a follow-up discussion,
it came to light that ROUND was not handling the unexpected case where the
first parameter is an empty cell in the same manner that Excel does.
Subsequent investigation showed that a boolean first parameter is permitted.
I broadened my investigation to include the following related functions.
- ROUNDUP
- ROUNDDOWN
- MROUND
- TRUNC
- INT
- FLOOR
- FLOOR.MATH
- FLOOR.PRECISE
- CEILING
- CEILING.MATH
- CEILING.PRECISE
All of these allow a NULL first parameter, and all except MROUND allow boolean.
For completeness, I will note that all treat null string as invalid.
I suspect there are other functions which permit
similarly unexpected parameters, but I consider them out of scope for this PR.
CEILING.MATH and CEILING.PRECISE were unimplemented, and are now supported
as part of this PR.
The tests for each of these functions have been re-coded, though all the original
test data is still included in the test cases, plus several new cases for each.
The new tests now take place as a user would invoke the functions,
through a spreadsheet cell rather than a
direct call to the appropriate function within Calculation/MathTrig.
Aside from being more realistic, the new tests are also more complete.
For example, FLOOR.MATH can take from 1-3 arguments, and the existing tests
confirmed that the function in Calculation could handle a single argument.
However, the function list in Calculation.php erroneously set the number of
arguments for FLOOR.MATH to exactly 3, so, if a user tried to get the calculated
result of a cell containing FLOOR.MATH(1.2), the result would be an Exception.
Aside from the parameter support, there are a few minor code changes.
Ods, as well as Gnumeric, allows the omission of the second parameter for
FLOAT and CEILING; Excel does not. A potential divide-by-zero error is
avoided in CEILING, FLOOR, and FLOORMATH.
I will note that it would probably be beneficial in terms of maintainability
to break MathTrig up into many individual modules. The same would hold for the
other Calculation modules. I would be willing to look into this if you agree
that it would be worthwhile.
* Problems Using Builtin PHP Functions Directly As Excel Functions
This fixes issue #1789.
As originally reported, stricter typing was causing PHP8 to throw
an exception when a non-numeric value was passed to the Round function.
Previous releases of PHP did not see this problem, however, on further
analysis, they were also incorrect in returning 0 as the result in the
erroneous situation, when they should have been returning a VALUE error.
Yet more analysis showed that other functions would also have problems,
and, in addition, might not handle invalid input (e.g. a negative length
passed to REPT) or output (e.g. NAN in the case of ACOS(2)) correctly.
The following MathTrig functions are affected:
ABS, ACOS, ACOSH, ASIN, ASINH, ATAN, ATANH,
COS, COSH, DEGREES (rad2deg), EXP, LN (log), LOG10,
RADIANS (deg2rad), REPT (str_repeat), SIN, SINH, SQRT, TAN, TANH.
One TextData function (REPT) is also affected.
This change lets PhpSpreadsheet validate the input for each of these
functions before passing control to the builtin, and handle the output
afterwards.
There were no explicit tests for any of these functions, a fact made
easy to ignore by the fact that PhpSpreadsheet delegated the heavy
lifting to PHP itself for these cases. A full suite of tests is
now added for each of the affected functions.
* Scrutinizer Recommendations
Only in 3 modules which are part of this PR.
* Improved Handling of Tan(PI/2)
Return DIV0 error for TAN when COS is very small.
* Additional Trig Tests
Results which should be infinity, i.e. DIV/0 error.
* Merge branch 'master' of C:\Projects\PHPOffice\PHPSpreadsheet\develop with conflicts.
* First pass at moving MathTrig tests into individual test files
* Appeasement to the great goddess PHPCS
* Appeasement to the great goddess PHPCS
* Minor scrutinizer issue resolved
* More refactoring of tests into individual test files fr each math/trig function
* More work on the math/trig test refactoring, plus a bit of tidyup of date/time tests as well
* Fix test
* Fix docblock in test
* Finish refactoring Math/Trig tests into separate files
* Fix SubTotal Test
* Import additional classes for SubTotal test
- Fix ISFORMULA() function to work with a cell reference to another worksheet
- Added calculation engine support for the new functions that were added in MS Excel 2013 and MS Excel 2016
- Text Functions
- CONCAT() Synonym for CONCATENATE()
- NUMBERVALUE() Converts text to a number, in a locale-independent way
- UNICHAR() Synonym for CHAR() in PHPSpreadsheet, which has always used UTF-8 internally
- UNIORD() Synonym for ORD() in PHPSpreadsheet, which has always used UTF-8 internally
- TEXTJOIN() Joins together two or more text strings, separated by a delimiter
- Logical Functions
- XOR() Returns a logical Exclusive Or of all arguments
- Date/Time Functions
- ISOWEEKNUM() Returns the ISO 8601 week number of the year for a given date
- Lookup and Reference Functions
- FORMULATEXT() Returns a formula as a string
- Engineering Functions
- ERF.PRECISE() Returns the error function integrated between 0 and a supplied limit
- ERFC.PRECISE() Synonym for ERFC
- Math and Trig Functions
- SEC() Returns the secant of an angle
- SECH() Returns the hyperbolic secant of an angle
- CSC() Returns the cosecant of an angle
- CSCH() Returns the hyperbolic cosecant of an angle
- COT() Returns the cotangent of an angle
- COTH() Returns the hyperbolic cotangent of an angle
- ACOT() Returns the cotangent of an angle
- ACOTH() Returns the hyperbolic cotangent of an angle
- Financial Functions
- PDURATION() Calculates the number of periods required for an investment to reach a specified value
- RRI() Calculates the interest rate required for an investment to grow to a specified future value
If $a or $b are not strings, you can get an error that fmod needs param 1 ($a) to be of type double.
MS Excel does not fall over when you insert an empty string in MOD, so I'm guessing PHPExcel should do the same?
MarkBaker
MarkBaker
oleibman
lucasnetau
oleibman
Martins Sipenko
Flinsch
Adrien Crivelli
n-longcape
Paul Kievits
paulkned
Ikko Ashimine
Fräntz Miccoli
marcusblevin
Sreten Ilić
Scorty
Matthias Van Woensel