Commit Graph

404 Commits

Author SHA1 Message Date
Mark Baker c699d144e2
Extract ACCRINT() and ACCRINTM() Financial functions into their own class (#1956)
* Extract ACCRINT() and ACCRINTM() Financial functions into their own class
Implement additional validations, with additional unit tests
Add support for the new calculation method argument for ACCRINT()
* Additional tests for Amortization functions
2021-03-26 22:49:16 +01:00
oleibman 9239b3deca
Continue MathTrig Breakup - Problem Children (#1954)
Continuing the process of breaking MathTrip.php up into smaller classes. This round takes care of all functions which might be an impediment to installing due to either uncovered code or "complexity":
- BASE
- FACT
- LCM
- MDETERM, MINVERSE, MMULT
- MULTINOMIAL
- PRODUCT
- QUOTIENT
- SERIESSUM
- SUM
- SUMPRODUCT

MathTrig and the members in directory MathTrig are now 100% covered. Many tests have been added, and some edge-case bugs are corrected. Some cases where PhpSpreadsheet had rejected numeric values stored as strings have been changed to accept them whenever Excel does; there had been no tests for that condition.

Boolean arguments are now accepted as arguments wherever Excel accpets them. Taking a cue from what has been done in Engineering, the parameter validation now happens in a routine which issues Exceptions for invalid values; this simplifies the code in the functions themselves. Thank you for doing that; I did not foresee how useful that was when I first looked at it.

Consistent with earlier changes of this nature, the versions in the MathTrig class remain, with a doc block indicating deprecation, and a stub call to the new routines.

All tests except for MINVERSE and MMULT are now handled in the context of a spreadsheet rather than a direct call to the calculation function which implements it. PhpSpreadsheet would need to handle dynamic arrays in order to test MINVERSE and MMULT in a spreadsheet context. Implementing that looks like it might be *very* challenging. It is not something I plan to look at, at least not in the near future.

One parsing problem turned up in the test conversion. It is in one of the SUMIF tests. It takes me to an area in Calculation where the comment says "I don't even want to know what you did to get here". It did not show up in the previous incarnation because, by using a direct call, the previous test managed to bypass the parsing. I have confirmed that this problem shows up in earlier releases of PhpSpreadsheet, so the changes in this PR did not cause it - they merely exposed it. I have left the test intact, but marked it "incomplete" for documentation purposes. I have not been able to get a handle on what's going wrong yet. I will probably open an issue on it if I can't resolve it soon. However, the test in question isn't a "real world" issue, and the error wasn't caused by this change, so I see no reason to delay this pending a resolution of the problem.

SUM had an idiosyncratic moment of its own. It had been ignoring non-numeric values, but Excel returns VALUE in that situation. So I changed it and wrote some new tests, which worked, but ... SUMIF uses several levels of indirection to get to SUM, and SUMIF *does* ignore non-numeric values, so a SUMIF test broke. SUM is a really simple function; the most practical approach seemed to be to clone it, with the string-accepting version being used by the Legacy version (which is called by SUMIF), and the non-string-accepting version being used in the Calculation Function table. That seems far easier and more practical than, for instance, adding a boolean parameter to the variable parameter list. As a follow-up, I will change SUMIF to explicitly call the appropriate new version, but I did not want to add that to this already large change.

SUM again - although it was fully covered beforehand, there was not a specific test member for it. There is now.

FACT had been coded to fail Gnumeric requests where the numeric argument has a decimal portion. However, Gnumeric does accept such an argument, and, unlike Excel and ODS, does not truncate it, but returns the result of a Gamma function call instead. This has been corrected.

When LCM included arguments which contained both 0 and a negative number, it returned 0 or NUM, whichever it found first. It is changed to always return NUM in that circumstance, as Excel does.

QUOTIENT had been documented as taking a variadic list of arguments. In fact, it takes exactly 2 - numerator and denominator - and the docblock and signature is fixed, even in the deprecated version.

The SERIESSUM docbock and signature are more accurate, even in the deprecated version. It is changed to ignore nulls, as Excel does, rather than return VALUE, and is one of the routines which previously rejected numbers in string form.

SUBTOTAL tests had used mocking for some reason. These are replaced with normal tests. And SUBTOTAL had a big surprise in store. That part of it which deals with hidden cells cares only whether the row is hidden, and doesn't care about the column's visibility.

I struggled with whether it should be SubTotal or Subtotal. I think the latter is correct, so that's how I proceeded. I don't think there are likely to be any other capitalization controversies.
2021-03-26 17:35:30 +09:00
Mark Baker 07ad800755
New Bessel Algorithm, providing a higher degree of accuracy and precision (#1946)
* New Bessel Algorithm, providing a higher degree of precision (12 decimal places) and still matching/exceeding MS Excel's precision across the range of values
2021-03-24 13:29:54 +01:00
Mark Baker 1a7b9a446a
First phase of refactoring the Excel Text functions (#1945)
* Refactoring the Excel Text functions
* More unit tests for utf-8 handling, for edge cases, and for argument validations
2021-03-23 13:34:28 +01:00
oleibman 9beacd21be
Complete Breakup Of Calculation/DateTime Functions (#1937)
* Complete Breakup Of Calculation/DateTime Functions

In conjunction with parallel breakups happening in other areas of Calculation, this change breaks up all the DateTime functions into their own classes. All methods remaining in DateTime itself have a doc block deprecation notice, and consist only of stub code to call the replacement methods. Coverage of DateTime itself and all the replacement methods is 100%.

There is only one substantive change to the code (see next paragraph). Among the non-substantive changes, it now adopts the same parsing technique (throwing and catching exceptions) already in use in Engineering and MathTrig. Boolean parameters are allowed in lieu of numbers when Excel allows them. Most of the code changes involve refactoring due to the need to avoid Scrutinizer "complexity" failures in what it will consider to be new methods.

Issue #1936 was opened just as I was staging this. It is now fixed. One existing WORKDAY test was wrong (noted in a comment in the test data file), and a bunch of new tests are added.

I found it confusing to use DateTime as a node of the the class name since most of the methods invoke native DateTime methods. So, everything is moved to directory DateTimeExcel, and that is what is used in the class names.

There are several follow-up activities that I am planning to undertake if this PR is merged.

- ODS supports dates well before 1900. There are exactly 2 assertions for this functionality. More are needed (and some functions might have to change to accept this).
- WEEKDAY has some poorly documented extra options for "style" which are not yet implemented.
- Most tests have been changed to use a formula as entered on a spreadsheet rather than a direct call to the method which implements the formula. There are 3 exceptions at this time. WORKDAY and NETWORKDAYS, which include arrays as part of their parameters, are more complicated than most. YEARFRAC was just too large to deal with now.
- There are direct calls to the now-deprecated methods in both source code and tests, mostly in Financial code, but possibly in others as well. These need to be changed.
- Some constants, none "officially" documented, remain in the original class. These should be either deleted or marked deprecated. I wasn't sure if deprecation was even possible (or desirable), and did not want that to be something which would cause Scrutinizer to fail the change.

* Deprecate Now-unused Constants, Fix Yearfrac bug, Change 3 Tests

Add new DateTime/Constants class, initially populated with constants used in Weeknum.

MS has another inconsistency with how it handles null cells in Yearfrac. Change PhpSpreadsheet to behave compatibly with this bug.

I have modified YearFrac, WorkDay, and NetworkDays tests to be more to my liking. Many tests added to YearFrac because of the bug above. Only minor modifications to the existing tests for the others.
2021-03-21 09:12:05 +01:00
Mark Baker d346318c2b
Start work on breaking down some of the Financial Excel functions (#1941)
* Start work on breaking down some of the Financial Excel functions
* Unhappy path unit tests for Treasury Bill functions
* Codebase for Treasury Bills includes logic for a different days between settlement and maturity calculation for OpenOffice; but Open/Libre Office now uses the Excel days calculation, so this discrepancy between packages is no longer required
* We've already converted the Settlement and Maturity dates to Excel timestamps, so there's no need to try doing it again when calculating the days between Settlement and Maturity
* Add Unit Tests for the Days per Year helper function
* Extract Interest Rate functions - EFFECT() and NOMINAL() - with additional validation, and unhappy path unit tests
* First pass at extracting the Coupon Excel functions
* Simplify the validation methods
* Extended unit tests to cover all combinations of frequency and basis, including leap years
Fix for COUPDAYSNC() when basis is US 360 and settlement date is the last day of the month
* Ensure that all Financial function code uses the new Helpers class for Days Per Year
2021-03-20 18:40:53 +01:00
Mark Baker 4cd6c7806e
Initial unit tests for Document Properties (#1932)
* Initial unit tests for Document Properties
* Typehinting in the document properties class
2021-03-17 18:36:13 +01:00
Mark Baker 09022256f4
Resolve Deprecated setMethods() call when Mocking for tests (#1925)
Resolve Deprecated `setMethods()` calls when Mocking for tests, using `onlyMethods()` and `addMethods()` instead
2021-03-15 14:50:05 +01:00
Mark Baker ae2468426f
jpgraph seems to be finally dying with PHP. (#1926)
* jpgraph seems to be finally dying with PHP. Until we have a valid alternative, disabling this run for PHP because it errors

https://github.com/HuasoFoundries/jpgraph looks like a natural successor, but it isn't BC so it will require some work to integrate
2021-03-15 14:14:44 +01:00
oleibman 30c880b5e6
Bitwise Functions and 32-bit (#1900)
* Bitwise Functions and 32-bit

When running the test suite with 32-bit PHP, a failure was reported in BITLSHIFT.
In fact, all of the following are vulnerable to problems, and didn't report
any failures only because of a scarcity of tests:
- BITAND
- BITOR
- BITXOR
- BITRSHIFT
- BITLSHIFT

Those last 2 can be resolved fairly easily by using multiplication by a power of 2
rather than shifting. The first 3 are a tougher nut to crack, and I will continue
to think how they might best be approached. For now, I have added skippable tests
for each of them, which at least documents the problem.

Aside from adding many new tests, some bugs were correctd:
- The function list in Calculation.php pointed BITXOR to BITOR.
- All 5 functions allow null/false/true parameters.
- BIT*SHIFT shift amount must be numeric, can be negative, allows decimal portion
(which is truncated to integer), and has an absolute value limit of 53.
- Because BITRSHIFT allows negative shift amount, its result can overflow
(in which case return NAN).
- All 5 functions disallow negative parameters (except ...SHIFT second parameter).
This was coded, but the code had been thwarted by an earlier is_int test.

* Full Support for AND/OR/XOR on 32-bit

Previous version did not support operands 2**32 through 2**48.
2021-03-14 20:05:31 +01:00
oleibman d99a4a3fac
Improve Coverage of BIN2DEC etc. (#1902)
* Improve Coverage of BIN2DEC etc.

The following functions have some special handling
depending on the Calculation mode:
- BIN2DEC
- BIN2HEX
- BIN2OCT
- DEC2BIN
- DEC2HEX
- DEC2OCT
- HEX2BIN
- HEX2DEC
- HEX2OCT
- OCT2BIN
- OCT2DEC
- OCT2HEX

Ods accepts boolean for its numeric argument.
This had already been coded, but there were no tests for it.

Gnumeric allows the use of non-integer argument where Excel/Ods do not.
The existing code allowed this for certain functions but not for others.
Gnumeric consistently allows it, so there is no need for parameter
gnumericCheck in convertBase::ValidateValue.
Again, there were no tests for this.

There were some minor changes needed:
- In functions where you are allowed to specify the numnber of "places" in the
result, there is an upper bound of 10 which had not been enforced.
- Negative values were not handled correctly in some cases.
- There was at least one (avoidable) error on a 32-bit system.
- Some upper and lower bounds were not being enforced. In addition to enforcing
those, the bounds are now defined as class constants in ConvertDecimal.

Many tests have been added, so that Engineering is now almost 100% covered.
The exception is some BESSEL code. There have been some recent changes to
BESSEL which are not yet part of my fork, so I could not address those now.
However, I freely admit that, when I looked at the uncovered portion, it seemed
like it might be a difficult task, so I probably wouldn't have tackled it anyhow.
In particular, the uncovered code seemed to deal with very large numbers,
and, although PhpSpreadsheet and Excel both give very large results for these
conditions, their answers are not particularly close to each other. I think
we're dealing with resuts approaching infinity. More study is needed.
2021-03-14 20:04:50 +01:00
oleibman 7e071e8abc
Coverage for Helper/Samples (#1920)
* Coverage for Helper/Samples

I was perplexed by the fact that Helper/Samples seemed to be entirely uncovered when running the test suite, since I know all the samples are run as part of the test. I think that what must be happening is that the Helper code is invoked mostly as part of a Data Provider (and therefore not counted), not as part of the test proper (which would count). So, this change adds a small number of tests which result in Samples being 100% covered.

Covering one statement was tricky - simulating the inability to create a test directory. Mocking, a technique I have not used before, solves this problem admirably.

* Suggestions From Mark Baker

Tests changed from assertEquals to assertSame.

Added @covers annotation to test class.

Validate parameter for method being mocked.
2021-03-14 20:04:07 +01:00
Mark Baker ed62526aca
First step extracting INDIRECT() and OFFSET() to their own classes (#1921)
* First step extracting INDIRECT() and OFFSET() to their own classes
* Start building unit tests for OFFSET() and INDEX()
* Named ranges should be handled by the Calculation Engine, not by the implementation of the Excel INDIRECT() function
* When calling the calculation engine to get the range of cells to return, INDIRECT() and OFFSET() should use the instance of the calculation engine for the current workbook to benefit from cached results in that range

There's a couple of minor bugfixes in here; but it's basically just refactoring of the INDIRECT() and OFFSET() Excel functions into their own classes - still needs a lot of work on unit testing; and there's a lot more that could be improved in the code itself (including handling of the a1 flag for R1C1 format in INDIRECT()
2021-03-14 19:58:10 +01:00
oleibman 0ce8509a8c
Continue MathTrig Breakup - Trig Functions (#1905)
* Continue MathTrig Breakup - Trig Functions

Continuing the process of breaking MathTrip.php up into smaller classes.
This round takes care of the trig and hyperbolic functions, plus a few others.
- COS, COSH, ACOS, ACOSH
- COT, COTH, ACOT, ACOTH
- CSC, CSCH
- SEC, SECH
- SIN, SINH, ASIN, ASINH
- TAN, TANH, ATAN, ATANH, ATAN2
- EVEN
- ODD
- SIGN

There are no bug fixes in this PR, except that boolean arguments are now
accepted for all these functions, as they are for Excel.
Taking a cue from what has been done in Engineering, the parameter validation
now happens in a routine which issues Exceptions for invalid values;
this simplifies the code in the functions themselves.

Consistent with earlier changes of this nature, the versions in the
MathTrig class remain, with a doc block indicating deprecation,
and a stub call to the new routines.

I think several more iterations will be needed to break up MathTrig completely.
2021-03-13 12:06:30 +01:00
Mark Baker 2259de578b
Lookup ref further tests and examples (#1918)
* Extract LookupRef\INDEX() into index() method of LookupRef\Matrix class
Additional tests
* Bugfix for returning a column using INDEX()
* Some improvements to ROW() and COLUMN()
* Simplify some of the INDEX() logic, eliminating redundant code
2021-03-11 22:34:47 +01:00
Mark Baker 499ce61cf7
Unhappy path tests for FORMULATEXT() Function (#1915)
* Unhappy path tests
2021-03-10 22:38:41 +01:00
oleibman 13b62becdd
Fix for Issue #1887 - Lose Track of Selected Cells After Save (#1908)
* Fix for Issue #1887 - Lose Track of Selected Cells After Save

Issue #1887 reports that selected cells are lost after saving Xlsx. Testing indicates that this applies to the object in memory, though not to the saved spreadsheet.

Xlsx writer tries to save calculated values for cells which contain formulas. Calculation::_calculateFormulaValue issues a getStyle call merely to retrieve the quotePrefix property, which, if set, indicates that the cell does not contain a formula even though it looks like one. A side-effect of calls to getStyle is that selectedCell is updated. That is clearly accidental, and highly undesirable, in this case. Code is changed to save selectedCell before getStyle call and restore it afterwards.

The problem was reported only for Xlsx save. To be on the safe side, test is made for output formats of Xlsx, Xls, Ods, Html (which basically includes Pdf), and Csv. For all of those, the object in memory is tested after the save. For Xlsx and Xls, the saved file is also tested. It does not make sense to test the saved file for Csv and Html. It does make sense to test it for Ods, but the necessary support is not yet present in either the Ods Reader or Ods Writer - a project for another day.

* Move Logic Out of Calculation, Add Support for Ods ActiveSheet and SelectedCells

Mark Baker thought logic belonged in Worksheet, not Calculation.
I couldn't get it to work in Worksheet, but doing it in Cell works,
and that has already been used to preserve ActiveSheet over call to
getCalculatedValue, so this just extends that idea to SelectedCells.

Original tests could not completely support Ods because of a lack of support
for ActiveSheet and SelectedCells in Ods Reader and Writer.
There's a lot missing in Ods support, but a journey of 1000 miles ...
Those two particular concepts are now supported for Ods.
2021-03-10 21:23:08 +01:00
Mark Baker 70f372d88c
Start refactoring the Lookup and Reference functions (#1912)
* Start refactoring the Lookup and Reference functions
 - COLUMN(), COLUMNS(), ROW() and ROWS()
 - LOOKUP(), VLOOKUP() and HLOOKUP()
 - Refactor TRANSPOSE() and ADDRESS() functions into their own classes

* Additional unit tests
 - LOOKUP()
 - TRANSPOSE()
 - ADDRESS()
2021-03-10 21:18:33 +01:00
Mark Baker c4ed0ee7b0
Minor scrutinizer improvements (#1906)
* Minor scrutinizer improvements
* Minor typing improvements
2021-03-07 14:22:03 +01:00
Mark Baker 2d8c8c8ecf
Trend unit tests (#1899)
- Move TREND() functions into the Statistical Trends class
- Unit tests for TREND()
- Create Confidence class for Statistical Confidence functions
2021-03-06 22:50:19 +01:00
Mark Baker a79a4ddbab
Statistical refactoring - Confidence() and Trend() (#1898)
- Move TREND() functions into the Statistical Trends class
- Unit tests for TREND()
- Create Confidence class for Statistical Confidence functions, and the CONFIDENCE() method
2021-03-04 21:45:56 +01:00
Mark Baker d2a83b404a
Statistical trends additional functions and unit tests (#1896)
* PEARSON() and CORREL() are identical functions
* Unit tests for GROWTH() function
* Move GROWTH() function into Statistical\Trends Class
2021-03-03 23:18:56 +01:00
Patrick Brouwers 000e6088c9
Reverted Scrutinzer fix in Xslx Reader listWorksheetInfo (#1895) 2021-03-03 21:34:45 +01:00
oleibman 04e7c30758
Fix Two 32-bit Timestamp Problems, and Minor getFormattedValue Bug (#1891)
I ran the test suite using 32-bit PHP. There were 2 places where changes
were needed due to 32-bit timestamps.

Reader\\Xml.php was using strtotime as an intermediate step in converting
a string timestamp to an Excel timestamp. The XML file type stores pure timestamps
(i.e. no date portion) as, e.g., 1899-12-31T02:30:00.000, and that value
causes an error using strtotime on a 32-bit system. However, it is sufficient
to use that value in a DateTime constructor, and that will work for 32- and 64-bit.

There was no test for that particular cell, so I added one to the XML read test.
And that's when I discovered the getFormattedValue bug. The cell's format
is `hh":"mm":"ss`. The quotes around the colons are disrupting the formatting.
PhpSpreadsheet formats the cell by converting the Excel format
to a Php Date format, in this case `H\:m\:s`.
That's a problem,
since Excel thinks 'm' means *minutes*, but PHP thinks it means *months*.
This is not a problem when the colon is not quoted; there are ample tests for that.
I added my best guess as to how to recognize this situation,
changing `\:m` to `:i`. The XML read test
now succeeds, and no other tests were broken by this change.

Test Shared\\DateTest had one test where the expected result of converting to a
Unix timestamp exceeds 2**32. Since a Unix timestamp is strictly an int,
that test fails on a 32-bit system. In the discussion regarding recently merged
PR #1870, it was felt that the user base might still be using the functions
that convert to and from a timestamp. So, we should not drop this test, but,
since it cannot succeed on a 32-bit system, I changed it to be skipped
whenever the expected result exceeded PHP_INT_MAX. There are 3 "toTimestamp"
functions within that test. Only one of these had been affected, but I thought
it was a good idea to add additional tests to the others to demonstrate this
condition.

In the course of testing, I also discovered some 32-bit problems with
bitwise and base-conversion functions. I am preparing separate PRs to
deal with those.
2021-03-03 10:52:11 +01:00
Mark Baker 42e8680fc0
Statistics more unit tests (#1889)
* Additional unit tests
2021-03-02 18:01:39 +01:00
Mark Baker 2eaf9b53aa
Start splitting some of the basic Statistical functions out into separate classes (#1888)
* Start splitting some of the basic Statistical functions out into separate classes containing just a few similar functions

* Splitting some of the basic Statistical functions out into separate classes containing just a few similar functions - MAX(), MAXA(), MIN() and MINA()

* Splitting some more of the basic Statistical functions out into separate classes containing just a few similar functions - StandardDeviations and Variances
2021-03-02 09:07:28 +01:00
Mark Baker 1d6f36d8df
Initial Formula Translation tests (#1886)
* Initial Formula Translation tests
2021-02-28 13:18:51 +01:00
oleibman 80a20fc991
100% Coverage for Calculation/DateTime (#1870)
* 100% Coverage for Calculation/DateTime

The code in DateTime is now completely covered.
Along the way, some errors were discovered and corrected.
- The tests which have had to be changed at the start of every year are
replaced by more robust equivalents which do not require annual changes.
- Several places in the code where Gnumeric and OpenOffice were thought to differ
from Excel do not appear to have had any justification.
I have left a comment where such code has been removed.
- Use DateTime when possible rather than date, time, or strftime functions to avoid
potential Y2038 problems.
- Some impossible code has been removed, replaced by an explanatory comment.
- NETWORKDAYS had a bug when the start date was Sunday. There had been no tests
of this condition.
- Some functions allow boolean and null arguments where a number is expected.
This is more complicated than the equivalent situations in MathTrig because
the initial date for these calculations can be Day 1 rather than Day 0.
- More testing for dates from 1900-01-01 through the fictitious
everywhere-but-Excel 1900-01-29.
    - This showed that there is an additional Excel bug - Excel evaluates
WEEKNUM(emptycell) as 0, which is not a valid result for
WEEKNUM without a second argument.
PhpSpreadsheet now duplicates this bug.
    - There is a similar and even worse bug for 1904-01-01 in 1904 calculations.
Weeknum returns 0 for this,
but returns the correct value for arguments of 0 or null.
    - DATEVALUE should accept 1900-02-29 (sigh) and relatives.
PhpSpreadsheet now duplicates this bug.
- Testing bootstrap sets default timezone. This appears to be a relic from
the releases of PHP where the unwise decision, subsequenly reversed,
was made to issue messages for
"no default timezone is set" rather than just use a sensible default.
This was a disruptive setting for some of the tests I added.
There is only one test in the entire suite which is default-timezone-dependent.
Setting and resetting of default timezone is moved to that test
(Reader/ODS/ODSTest), and out of bootstrap.
- There had been no testing of NOW() function.
- DATEVALUE test had no tests for 1904 calendar and needs some.
- DATE test changed 1900/1904 calendar in use without restoring it.
- WEEKDAY test had no tests for 1904 calendar and needs some.
    - Which revealed a bug in Shared/Date (excelToDateTimeObject was not
recognizing 1904-01-01 as valid when 1904 calendar is in use).
    - And an additional bug in that legal 1904-calendar values in the 0.0-1.0
range yielded the same "wrong" answers as 1900-calendar (see "One note" below).
Also the comment for one of the calendar-1904 tests was wrong in attempting
to identify what time of day the fraction represented.

I had wanted to break this up into a set of smaller modules, a process already
started for Engineering and MathTrig.
However the number of source code changes was sufficient that I wanted
a clean delta for this request.
If it is merged, I will work on breaking it up afterwards.

One note - Shared/Date/excelToDateTimeObject, when calendar-1900 is in use,
returns an unexpected result if its argument is between 0 and 1,
which is nominally invalid for that calendar.
It uses a base-1970 calendar in that instance. That check is not justifiable
for calendar-1904, where values in that range are legal,
so I made the check specific to calendar-1900,
and adjusted 3 1904 unit test results accordingly. However, I have to admit that
I don't understand why that check should be made even for calendar-1900.
It certainly doesn't match anything that Excel does.
I would recommend scrapping that code altogether.
If agreed, I would do this as part of the break-up into smaller modules.

Another note -
more controversially, it is clear that PhpSpreadsheet needs to support
the Excel and PHP date formats. Although it requires further study,
I am not convinced that it needs to support Unix timestamp format.
Since that is a potential source of Y2038 problems on 32-bit systems,
I would like to open a PR to deprecate the use of that format.
Please let me know if you are aware of a valid reason to continue to support it.
2021-02-27 20:43:22 +01:00
Mark Baker 08673b5820
Initial experiments using the new Database query logic with Conditional Statistical Functions (#1880)
- Refactoring of the Statistical Conditional functions (`AVERAGEIF()`, `AVERAGEIFS()`, `COUNTIF()`, `COUNTIFS()`, `MAXIFS()` and `MINIFS()` to use the new Database functions codebase.
- Extended unit testing
- Fix handling for null values
- Fixes to wildcard text searches

There's still scope for further improvements to memory usage and performance; but for now the code is stable with all unit tests passing
2021-02-27 18:26:12 +01:00
oleibman cb23cca3ec
Avoid Duplicate Titles When Reading Multiple HTML Files (#1829)
This issue arose while researching issue #1823. The issue was not a bug;
it just required clarification to the author of how to use the software.
But, while researching, I discovered that loading html into 2
sheets of a spreadsheet has a problem if the html title tag is the same
for the 2 sheets. PhpSpreadsheet would be able to save the resulting file,
but Excel would not be able to read it properly because of the duplicate title.
The worksheet setTitle method allows for disambiguation is such a circumstance.
The html reader passed a parameter indicating "don't disambiguate", but I can't
see any harm in changing that to "disambiguate". An extremely simple fix,
with tests to back it up.
2021-02-27 15:10:04 +01:00
Mark Baker 25f7dcb9fd
Enable support for wildcard text searches in Excel Database functions (#1876)
* Enable support for wildcard text searches in Excel Database functions
2021-02-23 19:26:29 +01:00
Mark Baker 40a6dee0a4
Enable support for dates and percentages in Excel Database functions (#1875)
* Enable support for dates and percentages in Excel Database functions, and CountIf/AverageIf/etc
* Enable support for booleans in Excel Database functions
2021-02-22 20:40:40 +01:00
Mark Baker 3764f30354
Refactor the Excel Database functions; and rewrite the query building (#1871)
* Refactor the Excel Database functions; and rewrite the query building to fix a bug with complex multi-criteria queries that involve both AND and OR conditions
* Fix handling for empty cells and NULL values in searches
* Expand unit tests; and add TODOs for dates, percentages, and wildcard text comparisons
2021-02-22 12:46:57 +01:00
Mark Baker 1318b90330
Bugfix #1858; Apply stricter scoping rules to named range/cell access (#1866)
* Apply stricter scoping rules to named range/cell access via Worksheet object
* Additional unit tests
2021-02-19 22:03:50 +01:00
Mark Baker 409c05b542
Additional Unit Test Cases for Convert UoM (#1864)
* Additional Unit Test Cases
2021-02-19 08:41:26 +01:00
Mark Baker b269c26f6e
Advanced Value Binder improvements (#1863)
* Refactor times, and add unit tests
2021-02-18 23:14:14 +01:00
Mark Baker 5afda811c9
Advanced Value Binder Improvements (#1862)
Advanced Value Binder
 - Improved format checking/setting for fractions;
 - Better percentage checking;
 - Some minor refactoring;
 - Improved unit testing
2021-02-18 19:17:47 +01:00
Mark Baker 7c7b229041
Let's see what Scrutinizer makes of these changes (#1859)
* Let's see what Scrutinizer makes of these changes
2021-02-18 12:39:24 +01:00
oleibman a24ca09bd4
Support 'Forms' for ROMAN Function (#1828)
* Support 'Forms' for ROMAN Function

This seems like an exceptionally silly thing for MS to have implemented
(Wikipedia on Roman Numerals: "There is no indication this is anything
other than an invention by the programmer").
Nevertheless, we can, and therefore probably should, implement it.

Not that I can implement it by an algorithm - Excel describes the various extra
styles as "more concise", "more concise", "more concise", and "simplified".
Nevertheless, since the universe of potential calls is relatively small,
it can be implemented as a table of values where the new forms would return
a different value than "classic". This table is relatively large, so I have
put it its own member to avoid overhead when the function is needed.

* Move ROMAN To Its Own Class

See discussion in PR #1837

* PHP 8.1 Deprecations

PHP8.1 Unit tests failed. 1 line fixes are available for
- Shared/Font
- Shared/XMLWriter
- Style/Color
- Writer/HTML

The problem is that an error is also reported for a strcmp at
line 272 of Cell/Cell. Not only does that line not invoke strcmp,
there is no strcmp in all of Cell/Cell, so I don't know what to make
of the error message. Oh well, let's fix what can be fixed.

Still dealing with the mysterious PHP8.1 unit test failure in Cell\Cell,
which seems to have something to do with strcmp. The only uses of
strcmp that I can find in src/ are in Calculation. I can't find any
use of it in test/ or samples/. So, if this doesn't fix the problem,
I may have to give up.
2021-02-13 21:23:58 +01:00
oleibman cabcfaa522
ROUND Accepts null, false, and true as First Parameter (#1837)
* ROUND Accepts null, false, and true as First Parameter

Issue #1789 was addressed by PR #1799. In a follow-up discussion,
it came to light that ROUND was not handling the unexpected case where the
first parameter is an empty cell in the same manner that Excel does.
Subsequent investigation showed that a boolean first parameter is permitted.
I broadened my investigation to include the following related functions.
- ROUNDUP
- ROUNDDOWN
- MROUND
- TRUNC
- INT
- FLOOR
- FLOOR.MATH
- FLOOR.PRECISE
- CEILING
- CEILING.MATH
- CEILING.PRECISE

All of these allow a NULL first parameter, and all except MROUND allow boolean.
For completeness, I will note that all treat null string as invalid.
I suspect there are other functions which permit
similarly unexpected parameters, but I consider them out of scope for this PR.

CEILING.MATH and CEILING.PRECISE were unimplemented, and are now supported
as part of this PR.

The tests for each of these functions have been re-coded, though all the original
test data is still included in the test cases, plus several new cases for each.
The new tests now take place as a user would invoke the functions,
through a spreadsheet cell rather than a
direct call to the appropriate function within Calculation/MathTrig.
Aside from being more realistic, the new tests are also more complete.
For example, FLOOR.MATH can take from 1-3 arguments, and the existing tests
confirmed that the function in Calculation could handle a single argument.
However, the function list in Calculation.php erroneously set the number of
arguments for FLOOR.MATH to exactly 3, so, if a user tried to get the calculated
result of a cell containing FLOOR.MATH(1.2), the result would be an Exception.

Aside from the parameter support, there are a few minor code changes.
Ods, as well as Gnumeric, allows the omission of the second parameter for
FLOAT and CEILING; Excel does not. A potential divide-by-zero error is
avoided in CEILING, FLOOR, and FLOORMATH.

I will note that it would probably be beneficial in terms of maintainability
to break MathTrig up into many individual modules. The same would hold for the
other Calculation modules. I would be willing to look into this if you agree
that it would be worthwhile.
2021-02-13 21:00:08 +01:00
Mark Baker 42ecc270ec
Extract Permutation functions from the Statistical class into a dedicated Permutations class (#1851)
* Extract Permutation functions from the Statistical class into a dedicated Permutations class

Retain the original methods in the Statistical class as stubs for BC, but deprecate them. They will be removed for PHPSpreadsheet v2

Note that unit tests still point to the Statistical class stubs; these should be modified to use the Permutations class directly when the stubs are removed

Also provided a basic implementationof the PERMUTATIONA() Function
2021-02-13 15:35:07 +01:00
oleibman 2fac9ee2f7
Stacked Alignment - Use Class Constant Rather than Literal (#1716)
* Stacked Alignment - Use Class Constant Rather than Literal

PR #1580 defined constants for "stacked" alignment in cells.
Using those constants outside of Style/Alignment was beyond the
scope of the original PR, but I said I would get to it.
This PR replaces all uses of literal -165, and appropriate uses of
literal 255, with the named constants, and adds tests to make sure
that the changed code is covered in the test suite.
2021-02-03 23:53:04 +01:00
Darren Maczka c82ff2526c
Fix/chart axis titles (#1760)
* use axPos value to determine whether an axis title is mapped to the XaxisLabel or YaxisLabel

* update changelog

* Fix php-cs-fixer violations

Co-authored-by: Darren Maczka <dkm@utk.edu>
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
2021-01-31 19:13:50 +01:00
Darren Maczka 44248cd04e
Fix/sheets xlsx chart (#1761)
* Add support for Google Sheets Exported XLSX Charts

Google Sheets XLSX charts use oneCellAnchor positioning and the data series
do not have the *Cache elements with cached values.

* update CHANGELOG

* Add support for Google Sheets Exported XLSX Charts

Google Sheets XLSX charts use oneCellAnchor positioning and the data series
do not have the *Cache elements with cached values. Because the reader had been
assuming *Cache elements existed as children of strRef and numRef, errors about
the node being deleted were thrown when reading Xlsx exported from Google Sheets.

Co-authored-by: Darren Maczka <dkm@utk.edu>
2021-01-31 18:53:54 +01:00
Alexander Gunkel fdc8e8d17a
Fix/1674 (#1688)
* Treat inline strings like strings in Open Document because it has no specific inline-string format

* implement data-type error

Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
2021-01-31 18:26:50 +01:00
Mark Baker 18abae7245
Additional statistical unit tests (#1818) 2021-01-31 16:42:17 +01:00
Mark Baker 5d309e982c
Extract remaining Excel function unit tests into separate test classes for each function (#1817)
* Extract remaining Financial function unit tests into separate test classes for each function

This makes it easier to manage unit tests if they are individual files rather than all in a single file

It also provides a stepping stone toward making it easier to test Excel functions when Excel errors no longer return a string, but an actual Excel exception that can be handled more cleanly
2021-01-31 15:09:56 +01:00
Mark Baker 41c8a4f1b4
Additional unit tests for previously untested financial functions (#1815)
* Additional unit tests for previously untested financial functions, and some additions to follow untested paths
* Start splitting Financial function tests out from the large FinancialTests class into individual test classes for each function
2021-01-30 18:45:31 +01:00
Mark Baker 4092da0525
Additional unit tests for statistical functions, with a fix to ordering for RANK() (#1813)
* Additional unit tests for statistical functions, with a fix to ordering for RANK()
2021-01-29 22:21:55 +01:00
もりもと たかひろ 8d2d78334f
Support DataBar of conditional formatting rule (#1754)
Implemented the databar of Conditional Type for XLSX Files.
- DataBar can be read, written, and added for basic use.
- Supports reading, writing and adding using "extLst".

About "extLst"
- https://docs.microsoft.com/en-us/openspecs/office_standards/ms-xlsx/07d607af-5618-4ca2-b683-6a78dc0d9627

The following setting items on the Excel setting screen can be read, written, and added.
- (minimum, maximum)type: Automatic, LowestValue, Number, Percent, Formula, Percentile
- Direction: context, leftToRight, rightToLeft (show data bar only)
- Fills Solid, Gradient
- FillColor: PositiveValues, NegativeValues
- Borders: Solid, None
- BorderColor: PositiveValues, NegativeValues
- Axis position: Automatic, Midpoint, None
- Axis color
2021-01-29 16:57:40 +01:00