* Keep Calculated String Results Below 32K
This is the result of an investigation into issue #2884 (see also PR #2913). It is, unfortunately, not a fix for the original problem; see the discussion in that PR for why I don't think there is a practical fix for that specific problem at this time.
Excel limits strings to 32,767 characters. We already truncate strings to that length when added to the spreadsheet. However, we have been able to exceed that length as a result of the concatenation operator (Excel truncates); as a result of the CONCATENATE or TEXTJOIN functions (Excel returns #CALC!); or as a result of the REPLACE, REPT, SUBSTITUTE functions (Excel returns #VALUE!). This PR changes PhpSpreadsheet to return the same value as Excel in these cases. Note that Excel2003 truncates in all those cases; I don't think there is a way to differentiate that behavior in PhpSpreadsheet.
However, LibreOffice and Gnumeric do not have that limit; if they have a limit at all, it is much higher. It would be fairly easy to use existing settings to differentiate between Excel and LibreOffice/Gnumeric in this respect. I have not done so in this PR because I am not sure how useful that is, and I can easily see it leading to problems (read in a LibreOffice spreadsheet with a 33K cell and then output to an Excel spreadsheet). Perhaps it should be handled with an additional opt-in setting.
I changed the maximum size from a literal to a constant in the one place where it was already being enforced (Cell/DataType). I am not sure that is the best place for it to be defined; I am open to suggestions.
* Implement Some Suggestions
... from @MarkBaker.
* PHP8.1 Deprecation Passing Null to String Function
For each of the files in this PR, one or more statements can pass a null to string functions like strlower. This is deprecated in PHP8.1, and, when deprecated messages are enabled, causes many tests to error out. In every case, use coercion to pass null string rather than null.
* TextData - Minor Changes, Test Coverage
Per agreement on a previous push, I looked into standardizing the initialization of the TextData functions (like Engineering and MathTrig), with particular regard for avoiding multiple later null coercions. This simplifies the code quite a bit. This PR also increases coverage to 100% for all TextData modules. All entries in Phpstan baseline for non-deprecated TEXTDATA functions are removed. There were some minor bugfixes.
Whereas Excel (and Gnumeric) treat booleans when supplied as strings as 'TRUE' or 'FALSE', ODS treats them as '1' or '0'. Unlike Excel, ODS generally does not allow bool for int arguments; it does, however, allow them for FIND and SEARCH. ODS allows boolean for into for SUBSTITUTE even though Excel doesn't. ODS allows bool for string for NUMBERVALUE and VALUE even though Excel doesn't. ODS accepts 0 as an argument for CHAR; Excel doesn't. Most of this seems like random decisions on the part of the developers; I've done my best to follow the products in each case. There is a new test member devoted to ODS tests.
Gnumeric has an anomaly vis-a-vis the others - if length is supplied to LEFT/MID/RIGHT as null, Gnumeric treats it as 0 rather than 1.
All tests now take place in the context of a spreadsheet ...
Except for RETURNSTRING, which is not the implementation of an Excel function, and is referred to in the rest of PhpSpreadsheet only in the unit tests for itself. It should probably be deprecated, but that is not part of this PR, just in case there is some reason for it that I couldn't discern.
I have tried to make the first line of each doc block identify the Excel function name rather than its name in PhpSpreadsheet. I think it makes things more comprehensible.
Some tests call Settings::setLocale, but there was no Settings::getLocale. At the end of the tests which do it, they invoke setLocale('EN-US'), which, in a practical sense, is sufficient. However, in theory it would be better for them to get the current locale before changing it, then changing it back to the original when the time came. I have added getLocale and made the appropriate testing change.
The CHAR function took an interesting turn. One can set the value of a cell to, say, CHAR(2), the ASCII/UTF-8 representation of a control character, which is not legal in certain contexts. The only Reader/Writer that could handle this without problems is Xls, which deals with binary data all the time. However, if you tried to write it to Xlsx, Excel would not be able to open the resulting file because of what it considers an illegal character. I changed the Xlsx writer to escape such characters when writing the value of a string function. I did not make any other changes to the Xlsx writer - it seems to me that setting a cell to CHAR(2) is legitimate, but setting it to say `"\x02"` seems less likely to be legitimate, so the latter will still fail (although `="\x02"` should work). The Xlsx reader already supports the escape mechanism that I added to the writer.
CHAR control character and Ods - not supported by either Reader or Writer. I did not attempt to add this now. There is lots still missing from ODS, and this item just can't be a high priority amongst all of those.
CHAR control character and Csv - it is supported by reader and writer if the file has a csv extension. However, trying to guess the mime type without an extension - the control character makes mime_get_type guess application/octet-stream, and PhpSpreadsheet therefore thinks that Csv can't read it.
CHAR control character and Html. Actual use of the control character in the file is subject to the same problems as Xml (i.e. Xlsx and Ods). It wasn't terribly difficult to get the Html Writer to change `"\x02"` to "``". I believe that this is technically legal; however, DOMDocument.loadHTML rejects it as an illegal entity, and I am not convinced that it is wrong to do so, so I haven't changed the Html writer.
* Scrutinizer
Correct 3 minor errors.
* First steps toward refactoring Statistical Distributions into smaller classes: BETA() and GAMMA() (and related functions) to start with... they all need a lot of tidying up, and more testing; but it's a start
* Add basic datatype validations to Beta and Gamma Excel function implementations
* Switch to using a trait with the validation methods to provide easier sharing between distribution classes
* Additional unit tests for Beta and Gamma functions, including unhappy path for validations
* Extract ChiSquared functions
* Additional argument validation checks with unit tests for Chi Squared functions
* Extract Fisher
* Move MEDIAN() and MODE() to the Averages class
* Extract filters for Median and Mode for common usage
* Problems Using Builtin PHP Functions Directly As Excel Functions
This fixes issue #1789.
As originally reported, stricter typing was causing PHP8 to throw
an exception when a non-numeric value was passed to the Round function.
Previous releases of PHP did not see this problem, however, on further
analysis, they were also incorrect in returning 0 as the result in the
erroneous situation, when they should have been returning a VALUE error.
Yet more analysis showed that other functions would also have problems,
and, in addition, might not handle invalid input (e.g. a negative length
passed to REPT) or output (e.g. NAN in the case of ACOS(2)) correctly.
The following MathTrig functions are affected:
ABS, ACOS, ACOSH, ASIN, ASINH, ATAN, ATANH,
COS, COSH, DEGREES (rad2deg), EXP, LN (log), LOG10,
RADIANS (deg2rad), REPT (str_repeat), SIN, SINH, SQRT, TAN, TANH.
One TextData function (REPT) is also affected.
This change lets PhpSpreadsheet validate the input for each of these
functions before passing control to the builtin, and handle the output
afterwards.
There were no explicit tests for any of these functions, a fact made
easy to ignore by the fact that PhpSpreadsheet delegated the heavy
lifting to PHP itself for these cases. A full suite of tests is
now added for each of the affected functions.
* Scrutinizer Recommendations
Only in 3 modules which are part of this PR.
* Improved Handling of Tan(PI/2)
Return DIV0 error for TAN when COS is very small.
* Additional Trig Tests
Results which should be infinity, i.e. DIV/0 error.
This had been intended to get 100% coverage for TextData functions, and it does that.
However, some minor bugs requiring source changes arose during testing.
- the Excel CHAR function restricts its argument to 1-255. PhpSpreadsheet CHARACTER
had been allowing 0+. Also, there is no need to test if iconv exists,
since it is part of Composer requirements.
- The DOLLAR function had been returning NUM for invalid arguments. Excel returns VALUE.
Also, negative amounts were not being handled correctly.
- The FIXEDFORMAT function had been returning NUM for invalid arguments. Excel FIXED returns VALUE.
- Fix ISFORMULA() function to work with a cell reference to another worksheet
- Added calculation engine support for the new functions that were added in MS Excel 2013 and MS Excel 2016
- Text Functions
- CONCAT() Synonym for CONCATENATE()
- NUMBERVALUE() Converts text to a number, in a locale-independent way
- UNICHAR() Synonym for CHAR() in PHPSpreadsheet, which has always used UTF-8 internally
- UNIORD() Synonym for ORD() in PHPSpreadsheet, which has always used UTF-8 internally
- TEXTJOIN() Joins together two or more text strings, separated by a delimiter
- Logical Functions
- XOR() Returns a logical Exclusive Or of all arguments
- Date/Time Functions
- ISOWEEKNUM() Returns the ISO 8601 week number of the year for a given date
- Lookup and Reference Functions
- FORMULATEXT() Returns a formula as a string
- Engineering Functions
- ERF.PRECISE() Returns the error function integrated between 0 and a supplied limit
- ERFC.PRECISE() Synonym for ERFC
- Math and Trig Functions
- SEC() Returns the secant of an angle
- SECH() Returns the hyperbolic secant of an angle
- CSC() Returns the cosecant of an angle
- CSCH() Returns the hyperbolic cosecant of an angle
- COT() Returns the cotangent of an angle
- COTH() Returns the hyperbolic cotangent of an angle
- ACOT() Returns the cotangent of an angle
- ACOTH() Returns the hyperbolic cotangent of an angle
- Financial Functions
- PDURATION() Calculates the number of periods required for an investment to reach a specified value
- RRI() Calculates the interest rate required for an investment to grow to a specified future value
By design, UTF-8 allows any byte-oriented substring searching algorithm,
since the sequence of bytes for a character cannot occur anywhere else
([source](https://en.wikipedia.org/wiki/UTF-8#Advantages_3)).
So `str_replace()` also works for UTF-8-encoded strings, assuming that
the input strings are valid UTF-8 strings. The previous implementation
of mbStrReplace() did nothing to detect invalid strings.
Also, `str_replace()` does not support [Unicode equivalence](https://en.wikipedia.org/wiki/Unicode_equivalence),
but nor do the other `mb_string` functions, and nor does `=SUBSTITUTE()` in Excel
(tested on Excel for Mac version 15.19.1, Excel 2016 for Windows and LibreOffice 5.1).
Closes#109
MarkBaker
oleibman
oleibman
MarkBaker
Adrien Crivelli
Rolands Usāns
Christian Schmidt