* 100% Coverage for Calculation/DateTime
The code in DateTime is now completely covered.
Along the way, some errors were discovered and corrected.
- The tests which have had to be changed at the start of every year are
replaced by more robust equivalents which do not require annual changes.
- Several places in the code where Gnumeric and OpenOffice were thought to differ
from Excel do not appear to have had any justification.
I have left a comment where such code has been removed.
- Use DateTime when possible rather than date, time, or strftime functions to avoid
potential Y2038 problems.
- Some impossible code has been removed, replaced by an explanatory comment.
- NETWORKDAYS had a bug when the start date was Sunday. There had been no tests
of this condition.
- Some functions allow boolean and null arguments where a number is expected.
This is more complicated than the equivalent situations in MathTrig because
the initial date for these calculations can be Day 1 rather than Day 0.
- More testing for dates from 1900-01-01 through the fictitious
everywhere-but-Excel 1900-01-29.
- This showed that there is an additional Excel bug - Excel evaluates
WEEKNUM(emptycell) as 0, which is not a valid result for
WEEKNUM without a second argument.
PhpSpreadsheet now duplicates this bug.
- There is a similar and even worse bug for 1904-01-01 in 1904 calculations.
Weeknum returns 0 for this,
but returns the correct value for arguments of 0 or null.
- DATEVALUE should accept 1900-02-29 (sigh) and relatives.
PhpSpreadsheet now duplicates this bug.
- Testing bootstrap sets default timezone. This appears to be a relic from
the releases of PHP where the unwise decision, subsequenly reversed,
was made to issue messages for
"no default timezone is set" rather than just use a sensible default.
This was a disruptive setting for some of the tests I added.
There is only one test in the entire suite which is default-timezone-dependent.
Setting and resetting of default timezone is moved to that test
(Reader/ODS/ODSTest), and out of bootstrap.
- There had been no testing of NOW() function.
- DATEVALUE test had no tests for 1904 calendar and needs some.
- DATE test changed 1900/1904 calendar in use without restoring it.
- WEEKDAY test had no tests for 1904 calendar and needs some.
- Which revealed a bug in Shared/Date (excelToDateTimeObject was not
recognizing 1904-01-01 as valid when 1904 calendar is in use).
- And an additional bug in that legal 1904-calendar values in the 0.0-1.0
range yielded the same "wrong" answers as 1900-calendar (see "One note" below).
Also the comment for one of the calendar-1904 tests was wrong in attempting
to identify what time of day the fraction represented.
I had wanted to break this up into a set of smaller modules, a process already
started for Engineering and MathTrig.
However the number of source code changes was sufficient that I wanted
a clean delta for this request.
If it is merged, I will work on breaking it up afterwards.
One note - Shared/Date/excelToDateTimeObject, when calendar-1900 is in use,
returns an unexpected result if its argument is between 0 and 1,
which is nominally invalid for that calendar.
It uses a base-1970 calendar in that instance. That check is not justifiable
for calendar-1904, where values in that range are legal,
so I made the check specific to calendar-1900,
and adjusted 3 1904 unit test results accordingly. However, I have to admit that
I don't understand why that check should be made even for calendar-1900.
It certainly doesn't match anything that Excel does.
I would recommend scrapping that code altogether.
If agreed, I would do this as part of the break-up into smaller modules.
Another note -
more controversially, it is clear that PhpSpreadsheet needs to support
the Excel and PHP date formats. Although it requires further study,
I am not convinced that it needs to support Unix timestamp format.
Since that is a potential source of Y2038 problems on 32-bit systems,
I would like to open a PR to deprecate the use of that format.
Please let me know if you are aware of a valid reason to continue to support it.
- Refactoring of the Statistical Conditional functions (`AVERAGEIF()`, `AVERAGEIFS()`, `COUNTIF()`, `COUNTIFS()`, `MAXIFS()` and `MINIFS()` to use the new Database functions codebase.
- Extended unit testing
- Fix handling for null values
- Fixes to wildcard text searches
There's still scope for further improvements to memory usage and performance; but for now the code is stable with all unit tests passing
This issue arose while researching issue #1823. The issue was not a bug;
it just required clarification to the author of how to use the software.
But, while researching, I discovered that loading html into 2
sheets of a spreadsheet has a problem if the html title tag is the same
for the 2 sheets. PhpSpreadsheet would be able to save the resulting file,
but Excel would not be able to read it properly because of the duplicate title.
The worksheet setTitle method allows for disambiguation is such a circumstance.
The html reader passed a parameter indicating "don't disambiguate", but I can't
see any harm in changing that to "disambiguate". An extremely simple fix,
with tests to back it up.
* Enable support for dates and percentages in Excel Database functions, and CountIf/AverageIf/etc
* Enable support for booleans in Excel Database functions
* Refactor the Excel Database functions; and rewrite the query building to fix a bug with complex multi-criteria queries that involve both AND and OR conditions
* Fix handling for empty cells and NULL values in searches
* Expand unit tests; and add TODOs for dates, percentages, and wildcard text comparisons
Advanced Value Binder
- Improved format checking/setting for fractions;
- Better percentage checking;
- Some minor refactoring;
- Improved unit testing
* Support 'Forms' for ROMAN Function
This seems like an exceptionally silly thing for MS to have implemented
(Wikipedia on Roman Numerals: "There is no indication this is anything
other than an invention by the programmer").
Nevertheless, we can, and therefore probably should, implement it.
Not that I can implement it by an algorithm - Excel describes the various extra
styles as "more concise", "more concise", "more concise", and "simplified".
Nevertheless, since the universe of potential calls is relatively small,
it can be implemented as a table of values where the new forms would return
a different value than "classic". This table is relatively large, so I have
put it its own member to avoid overhead when the function is needed.
* Move ROMAN To Its Own Class
See discussion in PR #1837
* PHP 8.1 Deprecations
PHP8.1 Unit tests failed. 1 line fixes are available for
- Shared/Font
- Shared/XMLWriter
- Style/Color
- Writer/HTML
The problem is that an error is also reported for a strcmp at
line 272 of Cell/Cell. Not only does that line not invoke strcmp,
there is no strcmp in all of Cell/Cell, so I don't know what to make
of the error message. Oh well, let's fix what can be fixed.
Still dealing with the mysterious PHP8.1 unit test failure in Cell\Cell,
which seems to have something to do with strcmp. The only uses of
strcmp that I can find in src/ are in Calculation. I can't find any
use of it in test/ or samples/. So, if this doesn't fix the problem,
I may have to give up.
* ROUND Accepts null, false, and true as First Parameter
Issue #1789 was addressed by PR #1799. In a follow-up discussion,
it came to light that ROUND was not handling the unexpected case where the
first parameter is an empty cell in the same manner that Excel does.
Subsequent investigation showed that a boolean first parameter is permitted.
I broadened my investigation to include the following related functions.
- ROUNDUP
- ROUNDDOWN
- MROUND
- TRUNC
- INT
- FLOOR
- FLOOR.MATH
- FLOOR.PRECISE
- CEILING
- CEILING.MATH
- CEILING.PRECISE
All of these allow a NULL first parameter, and all except MROUND allow boolean.
For completeness, I will note that all treat null string as invalid.
I suspect there are other functions which permit
similarly unexpected parameters, but I consider them out of scope for this PR.
CEILING.MATH and CEILING.PRECISE were unimplemented, and are now supported
as part of this PR.
The tests for each of these functions have been re-coded, though all the original
test data is still included in the test cases, plus several new cases for each.
The new tests now take place as a user would invoke the functions,
through a spreadsheet cell rather than a
direct call to the appropriate function within Calculation/MathTrig.
Aside from being more realistic, the new tests are also more complete.
For example, FLOOR.MATH can take from 1-3 arguments, and the existing tests
confirmed that the function in Calculation could handle a single argument.
However, the function list in Calculation.php erroneously set the number of
arguments for FLOOR.MATH to exactly 3, so, if a user tried to get the calculated
result of a cell containing FLOOR.MATH(1.2), the result would be an Exception.
Aside from the parameter support, there are a few minor code changes.
Ods, as well as Gnumeric, allows the omission of the second parameter for
FLOAT and CEILING; Excel does not. A potential divide-by-zero error is
avoided in CEILING, FLOOR, and FLOORMATH.
I will note that it would probably be beneficial in terms of maintainability
to break MathTrig up into many individual modules. The same would hold for the
other Calculation modules. I would be willing to look into this if you agree
that it would be worthwhile.
* Extract DELTA() and GESTEP() functions from the Engineering class into a dedicated Comparison classes
Retain the original methods in the Engineering class as stubs for BC, but deprecate them. They will be removed for PHPSpreadsheet v2
Note that unit tests still point to the Engineering class stubs; these should be modified to use the Erf and ErfC classes directly when the stubs are removed
* Extract Permutation functions from the Statistical class into a dedicated Permutations class
Retain the original methods in the Statistical class as stubs for BC, but deprecate them. They will be removed for PHPSpreadsheet v2
Note that unit tests still point to the Statistical class stubs; these should be modified to use the Permutations class directly when the stubs are removed
Also provided a basic implementationof the PERMUTATIONA() Function
* Stacked Alignment - Use Class Constant Rather than Literal
PR #1580 defined constants for "stacked" alignment in cells.
Using those constants outside of Style/Alignment was beyond the
scope of the original PR, but I said I would get to it.
This PR replaces all uses of literal -165, and appropriate uses of
literal 255, with the named constants, and adds tests to make sure
that the changed code is covered in the test suite.
* use axPos value to determine whether an axis title is mapped to the XaxisLabel or YaxisLabel
* update changelog
* Fix php-cs-fixer violations
Co-authored-by: Darren Maczka <dkm@utk.edu>
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
* Add support for Google Sheets Exported XLSX Charts
Google Sheets XLSX charts use oneCellAnchor positioning and the data series
do not have the *Cache elements with cached values.
* update CHANGELOG
* Add support for Google Sheets Exported XLSX Charts
Google Sheets XLSX charts use oneCellAnchor positioning and the data series
do not have the *Cache elements with cached values. Because the reader had been
assuming *Cache elements existed as children of strRef and numRef, errors about
the node being deleted were thrown when reading Xlsx exported from Google Sheets.
Co-authored-by: Darren Maczka <dkm@utk.edu>
* Treat inline strings like strings in Open Document because it has no specific inline-string format
* implement data-type error
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
* Extract remaining Financial function unit tests into separate test classes for each function
This makes it easier to manage unit tests if they are individual files rather than all in a single file
It also provides a stepping stone toward making it easier to test Excel functions when Excel errors no longer return a string, but an actual Excel exception that can be handled more cleanly
* Additional unit tests for previously untested financial functions, and some additions to follow untested paths
* Start splitting Financial function tests out from the large FinancialTests class into individual test classes for each function
* Revert "Fix cant get right format chinese date format error"
This reverts commit 8c58385d6c.
* formatAsDate strip language metadata (fixes#1616)
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
Implemented the databar of Conditional Type for XLSX Files.
- DataBar can be read, written, and added for basic use.
- Supports reading, writing and adding using "extLst".
About "extLst"
- https://docs.microsoft.com/en-us/openspecs/office_standards/ms-xlsx/07d607af-5618-4ca2-b683-6a78dc0d9627
The following setting items on the Excel setting screen can be read, written, and added.
- (minimum, maximum)type: Automatic, LowestValue, Number, Percent, Formula, Percentile
- Direction: context, leftToRight, rightToLeft (show data bar only)
- Fills Solid, Gradient
- FillColor: PositiveValues, NegativeValues
- Borders: Solid, None
- BorderColor: PositiveValues, NegativeValues
- Axis position: Automatic, Midpoint, None
- Axis color
* Delete Temporary Files In XssVulnerabilityTest
They need not exist after the test. Some of them are placed in
current directory, which means Git thinks they are needed.
* Problems Using Builtin PHP Functions Directly As Excel Functions
This fixes issue #1789.
As originally reported, stricter typing was causing PHP8 to throw
an exception when a non-numeric value was passed to the Round function.
Previous releases of PHP did not see this problem, however, on further
analysis, they were also incorrect in returning 0 as the result in the
erroneous situation, when they should have been returning a VALUE error.
Yet more analysis showed that other functions would also have problems,
and, in addition, might not handle invalid input (e.g. a negative length
passed to REPT) or output (e.g. NAN in the case of ACOS(2)) correctly.
The following MathTrig functions are affected:
ABS, ACOS, ACOSH, ASIN, ASINH, ATAN, ATANH,
COS, COSH, DEGREES (rad2deg), EXP, LN (log), LOG10,
RADIANS (deg2rad), REPT (str_repeat), SIN, SINH, SQRT, TAN, TANH.
One TextData function (REPT) is also affected.
This change lets PhpSpreadsheet validate the input for each of these
functions before passing control to the builtin, and handle the output
afterwards.
There were no explicit tests for any of these functions, a fact made
easy to ignore by the fact that PhpSpreadsheet delegated the heavy
lifting to PHP itself for these cases. A full suite of tests is
now added for each of the affected functions.
* Scrutinizer Recommendations
Only in 3 modules which are part of this PR.
* Improved Handling of Tan(PI/2)
Return DIV0 error for TAN when COS is very small.
* Additional Trig Tests
Results which should be infinity, i.e. DIV/0 error.
Now supports all current UoM in all categories, with both 1- and 2-character multiplier prefixes, and binary multiplier prefixes, including the new Temperature scales
* CSV - Guess Encoding, Handle Null-string Escape
This is in response to issue #1647 (detect CSV character encoding).
First, my tests with mb_detect_encoding indicate that it doesn't work
well enough; regardless, users can always do that on their own
if they deem it useful.
Rolling my own is also troublesome, but I can at least:
a. Check for BOM (UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, UTF-32LE).
b. Do some heuristic tests for each of the above encodings.
c. Fallback to a user-specified encoding (default CP1252)
if a and b don't yield result.
I think this is probably useful enough to include, and relatively
easy to expand if other potential encodings should be considered.
Starting with PHP7.4, fgetcsv allows specification of null string as
escape character in fgetcsv. This is a much better choice than the PHP
(and PhpSpreadsheet) default of backslash in that it handles the file
in the same manner as Excel does. There is one statement in Reader/CSV
which would be adversely affected if the caller so specified (building
a regular expression under the assumption that escape character is
a single character). Fix that statement appropriately and add tests.
Issue has been marked stale, but ...
Sylk read sets worksheet title to filename (minus .slk).
If that is >31 characters, PhpSpreadsheet throws Exception.
This change truncates sheet title, as Excel does, to 31 characters.
* Replace voku/anti-xss with ezyang/htmlpurifier. Despite anti-xss being a smaller footprint dependency, an a better license fit with our MIT license, there are issues with it's automatic it sanitisation of global variables causing side effects
* Additional unit tests for xss in html writer cell comments
This had been intended to get 100% coverage for TextData functions, and it does that.
However, some minor bugs requiring source changes arose during testing.
- the Excel CHAR function restricts its argument to 1-255. PhpSpreadsheet CHARACTER
had been allowing 0+. Also, there is no need to test if iconv exists,
since it is part of Composer requirements.
- The DOLLAR function had been returning NUM for invalid arguments. Excel returns VALUE.
Also, negative amounts were not being handled correctly.
- The FIXEDFORMAT function had been returning NUM for invalid arguments. Excel FIXED returns VALUE.
* Apply Column and Row Styles to Existing Cells
This is a fix for issue #1712.
When a style is applied to an entire row or column, it is currently
only effective for cells which don't already contain a value.
The code needs to iterate through existing cells in the row/column
in order to apply the style to them.
This could be considered a breaking change, however, I believe that
the change makes things operate as users would expect, and that the
existing implementation is incomplete.
The change also removes protected element conditionalStyles from
the Style class. That element is an unused remnant, and can no longer be
set or retrieved - methods getConditionalStyles and setConditionalStyles
actually act on an element in the Worksheet class.
Finally, additional tests are added so that Style, and in fact the
entire Style directory, now has 100% test coverage.
* Scrutinizer Changes
Scrutinizer flagged 6 statements. 5 can be easily corrected.
One is absolutely wrong (it thinks iterating through cells in column
can return null). Let's see if we can satisfy it.
* Remove Exception For CellIterator on Empty Row/Column
For my first attempt at this change, which corrects a bug by updating styles
for non-empty cells when a style is set on a row or column, I wished to make things
more efficient by using setIterateOnlyExistingCells, something which the
existing documentation recommends. This caused an exception to be generated
when the row or column is empty. So I removed that part of the change while I
researched what was going on.
I have completed that research. The existing code does throw an exception
when the row/column is empty and iterateOnlyExistingCells is true. However,
that does not seem like a reasonable action. This situation is analagous to
iterating over an empty array, and that action is legal and does not throw.
The same should apply here. There were no tests for this situation,
and now there are.
I have added additional tests, and coverage for all of RowCellIterator,
ColumnCellIterator, and CellIterator are all now 100%. Some of my new tests
were added in new members, because the existing tests all relied on mocking,
which was not the best choice for the new tests. One of the existing tests
for RowCellIteratorTest (testSeekOutOfRange) was wrong; it issued the expected
exception, but for the wrong reason. I have added an additional test to
ensure that it fails "correctly".
The existing documentation says that the default value for
IterateOnlyExistingCells is true. In fact, the default value is false.
I have corrected the documentation.
* More Scrutinizer
I believe its analysis is incorrect, but this should silence it.
* DocBlock Correction
ColumnCellIterator DocBlock for current indicated it could return null
or Cell, but it can really return only Cell. This had caused Scrutinizer
to complain earlier.
* PHP8 Environment Appears to be Fixed
Cosmetic change to Doc member. I suspect there is a way to rerun all
the tests without another push, but I have been unable to figure out how.
* Fix for 3 Issues Involving ReadXlsx and NamedRange
Issues #1686 and #1723, which provide sample spreadsheets, are probably
solved by this ticket. Issue #1730 is also probably solved, but I have
no way to verify.
There are two problems with how PhpSpreadsheet is handling things now.
Although the first problem is much less severe, and isn't really a factor
in the issues named above, it is helpful to get it out of the way first.
If you define a named range in Excel, and then delete the sheet where
the range exists, Excel saves the range as #REF!. If there is a cell which
references the range, it will similarly have the value #REF! when you open
the Excel file.
Currently, PhpSpreadsheet discards the #REF! definition, so a cell which
references the range will appear as #NAME? rather than #REF!.
This PR changes the behavior so that PhpSpreadsheet retains the #REF!
definition, and cells which reference it will appear as #REF!.
The second problem is the more severe, and is, I believe, responsible
for the 3 issues identified above.
If you define a named range and the sheet on which the range is defined
does not exist at the time, Excel will save the range as something like:
'[1]Unknown Sheet'!$A$1
If a cell references such a range, Excel will again display #REF!.
PhpSpreadsheet currently throws an Exception when it encounters
such a definition while reading the file. This PR changes
the behavior so that PhpSpreadsheet saves the definition as #REF!,
and cells which reference it will behave similarly.
For the record, I will note that Excel does not magically recalculate when a
missing sheet is subsequently added, despite the fact that the reference
might now become resolvable. PhpSpreadsheet behaves likewise.
* Remove Dead Code in Test
Identified it after push but before merge.
This is a fix for issue #1735.
It adds tests for this situation, and similar situations involving
adding new sheets and accessing existing ones.
Coverage for Spreadsheet.php increases from 69% to 75% as a result.
There are no changes to code. Additional tests are added,
so that the following 6 items now have 100% test coverage:
- Comment
- DefinedName
- DocumentGenerator
- IOFactory
- NamedFormula
- NamedRange
oleibman
Mark Baker
ElPopcorn
Mats Sibelius
Darren Maczka
Alexander Gunkel
SheetJSDev
もりもと たかひろ
Martins Sipenko
Gianluca Giovinazzo
Flinsch
Adrien Crivelli