4185aaed10
Dataframe deprecation ( #2353 )
2018-07-06 00:16:22 -07:00
b108419628
Cast locator with index type ( #2274 )
2018-06-21 08:28:22 -07:00
aa5daa1b82
fixing zero length partitions ( #2237 )
...
fixing bugs to fully handle zero len parts
resolve comments
renaming imports
Add getattr to groupby
testing
2018-06-21 08:25:32 -07:00
6bf48f47bc
addressing comments ( #2210 )
2018-06-20 16:24:37 -07:00
8560993b46
[Dataframe] Change pandas and ray.dataframe imports ( #1942 )
...
* fixing zero length partitions
* fixing bugs to fully handle zero len parts
* resolve comments
* renaming imports
2018-06-15 16:17:16 -07:00
b886ceca47
[DataFrame] Implement __array_wrap__ ( #2218 )
...
* Implement __array_wrap__
* Removing unnecessary test
2018-06-11 08:56:43 -07:00
c8c0349511
[DataFrame] Temporarily changing the requirement until our pandas compat is updated ( #2197 )
...
* Temporarily changing the requirement until our pandas compat is updated
for 0.23
* Fix lint
2018-06-06 12:01:43 -07:00
a139a5df8c
[DataFrame] Implement Memoizer ( #2157 )
...
* Implement Memoizer
* Add LRUCache
* Add comments
2018-06-05 07:18:12 -07:00
b56c8ed8dc
[DataFrame] Fix equals and make it more efficient ( #2186 )
...
* Fixing equals
* Adding test fix
* Working on fix for equals and drop
* Fix equals and fix tests to use ray.dataframe.equals
* Addressing comments
2018-06-04 13:10:06 -07:00
a5d888e49b
[DataFrames] More dtypes optimizations ( #2124 )
...
* Pass dtypes for some DataFrame constructors
* More optimizations with dtypes_cache
* Optimizations
2018-06-04 10:50:13 -07:00
eb1d7ac4bc
Add empty df test ( #1879 )
2018-05-27 09:25:50 -07:00
74cca3b284
[DataFrame] Fixing the code formatting of the tests ( #2123 )
...
* Fixing the code formatting of the tests
* Fixing tests and removing from_pandas
* Addressing comment
* Addressing comments
* Fix lint
2018-05-26 11:24:01 -07:00
68b11c8251
[DataFrame] Speed up dtypes ( #2118 )
...
* Don't recreate _block_partitions in _correct_dtypes
Further dtypes performance optimizations
Fix bugs
Redesign speedup
Address feedback
* Remove _correct_column_dtypes
2018-05-23 16:35:17 -07:00
4584193308
[DataFrame] Refactor GroupBy Methods and Implement Reindex ( #2101 )
...
* fix 1D blocks case
* Add groupby test code
* begin writing tests
* Fix bug on groupby(axis=1, ...)
* implement reindex
* fix index misalignment after groupby
* fix df.apply bug
* fix groupby.apply
* fix agg funcs
* finish groupby tests
* finish test suite for groupby
* fixing lint
* undo new line
* fix python2 index copy bug
* Concat Series into ray.df
* fixing python2 issues
* resolving all python 2 tests
* handle multiindex on apply
* resolve comments
* updating docstring
* fix lint
* fix lint again
* address comments
2018-05-22 16:34:07 -07:00
317c9450e7
[DataFrame] Test bugfixes ( #2111 )
2018-05-21 23:01:19 -07:00
f1fc373de7
[DataFrame] Update initializations of IndexMetadata which use outdated APIs ( #2103 )
...
* Update calls which use outdated APIs
* Fix lengths of IndexMetadata
2018-05-21 09:19:41 -10:00
f795173b51
Use flake8-comprehensions ( #1976 )
...
* Add flake8 to Travis
* Add flake8-comprehensions
[flake8 plugin](https://github.com/adamchainz/flake8-comprehensions ) that
checks for useless constructions.
* Use generators instead of lists where appropriate
A lot of the builtins can take in generators instead of lists.
This commit applies `flake8-comprehensions` to find them.
* Fix lint error
* Fix some string formatting
The rest can be fixed in another PR
* Fix compound literals syntax
This should probably be merged after #1963 .
* dict() -> {}
* Use dict literal syntax
dict(...) -> {...}
* Rewrite nested dicts
* Fix hanging indent
* Add missing import
* Add missing quote
* fmt
* Add missing whitespace
* rm duplicate pip install
This is already installed in another file.
* Fix indent
* move `merge_dicts` into utils
* Bring up to date with `master`
* Add automatic syntax upgrade
* rm pyupgrade
In case users want to still use it on their own, the upgrade-syn.sh script was
left in the `.travis` dir.
2018-05-20 16:15:06 -07:00
9e46de9830
[DataFrame] Update _inherit_docstrings ( #2085 )
...
* Update _inherit_docstrings
* Add groupby __init__
2018-05-18 17:50:41 -10:00
0b07602c89
[DataFrame] Refactor __delitem__ ( #2080 )
...
* Implement the bug fix
* Fix flake8
2018-05-18 08:58:20 -10:00
afbb260ca4
[DataFrame] Improve performance of iteration methods ( #2026 )
...
* fix iterrows
* make iteration methods performant
* resolving comments
* remove indexing from iterator
* switch to iterator syntax
2018-05-17 11:45:21 -10:00
ae17ebd032
[DataFrame] Implement to_csv ( #2014 )
...
* Add map, reduce, merge_dtypes
bug fixes
Unify dtypes on DataFrame creation
Formatting and comments
Cache dtypes
Fix bug in _merge_dtypes
Fix bug
Changed caching logic
Fix dtypes issue in read_csv
Invalidate dtypes cache when inserting column
Simplify unifying dtypes and improve caching
Fix typo
Better caching of dtypes
Fix merge conflicts
Implemented some to_csv functions
Support read_csv from buffers
Expose date_range, NaT, Timedelta from pandas
Add testing utils
Redirect imports to Pandas
Fix imports
Fix read_csv when index_col is specified
Update imports from Pandas
Fix bugs
Use util API
Fix nasty bug
Add missing import
Don't distribute reading of compressed files
Add test utilities for Pandas tests
Add test for to_csv
Add warnings
Fix rebase artifacts
* Fix rebase artifacts
* Fix bugs in read_csv indexing
* Fix bugs in read_csv
* Fix bug for IndexMetadata with _length 1
Remove testing imports
* Rebase artifacts and formatting
* Start to_csv without CSV formatter
* Fix bug in _map_partitions
* Initial implementation for improved to_csv
* Fix bug with insert
* Bugfixes
* Remove CSV Formatter
* Formatting
* Fix python2 bug
* Fix additional python2 issue
2018-05-17 11:35:17 -10:00
7549209aea
[DataFrame] Allows DataFrame constructor to take in another DataFrame ( #2072 )
...
* allow constructor to take in other DataFrame
* rename _data to _frame_data
2018-05-16 16:17:20 -10:00
825c227c2b
[DataFrame] Refactor indexers and implement setitem ( #2020 )
...
* Reset to_pandas change; update current
* Fix pd_df bug
2018-05-15 12:27:28 -07:00
8fbb88485b
Create RemoteFunction class, remove FunctionProperties, simplify worker Python code. ( #2052 )
...
* Cleaning up worker and actor code. Create remote function class. Remove FunctionProperties object.
* Remove register_actor_signatures function.
* Small cleanups.
* Fix linting.
* Support @ray.method syntax for actor methods.
* Fix pickling bug.
* Fix linting.
* Shorten testBlockingTasks.
* Small fixes.
* Call get_global_worker().
2018-05-14 14:35:23 -07:00
89e2eef3f3
[DataFrame] Fixing bugs in groupby ( #2031 )
2018-05-10 11:44:19 -07:00
b79912ec74
[DataFrame] Fixes dropna subset bug ( #2018 )
...
* fix dropna
* resolve comment
2018-05-10 08:25:24 -07:00
72a3a6cb02
[DataFrame] Implement where ( #1989 )
2018-05-09 14:05:52 -07:00
d2c193ed2c
[DataFrame] Add direct pandas imports for MVP ( #1960 )
...
* Add direct pandas imports for MVP
* rebase artifact
2018-05-08 19:19:32 -07:00
b1e32ca6c2
Fixing ascii error for Python2 ( #2009 )
2018-05-07 11:56:40 -07:00
1f82a46473
[DataFrame] Implements df.update ( #1997 )
...
* Working on fixing update
* Fixing update implementation
* Adding test
* Addressing comments
2018-05-07 08:55:40 -07:00
12da021717
[DataFrame] Implements df.as_matrix ( #2001 )
...
* Implement df.as_matrix
* Addressing comments
* Addressing comments
2018-05-06 23:36:39 -07:00
1848745223
[DataFrame] Implement quantile ( #1992 )
...
* added quantile method
* updated init for datetime signatures
* updated documentation for _map_partitions return type
* removed extraneous print call
* updated for simplicity
* fixed dtyping issues and error raising
* updated datetime dtype checking
* Fixing quantile implementation
* Fix minor bug
* Fixing diff
2018-05-06 18:25:13 -07:00
ad1afeb268
[DataFrame] Impement sort_values and sort_index ( #1977 )
2018-05-06 09:53:29 -07:00
9f28529e2c
[DataFrame] Implement rank ( #1991 )
...
* rank method completed
* added sanity checks
* flake8
* updated sanity checks
* flake8
* updated sanity checks and style
* updated dtype logic
* Fixing test
2018-05-06 09:32:33 -07:00
857458c37c
[DataFrame] Implemented prod, product, added test suite ( #1994 )
...
* implemented prod/product, modified declaration for sum, added pandas test suite
* fixed tests
* removed test_analytics file
* implemented nunique, skew
* fixed requested changes
* added nunique, skew
* fixed tests in request
* added newline back
* fixed newlines hopefully
* fixed flake8 issues
* more flake8 issues
* fixed test for prod
2018-05-05 21:25:42 -07:00
36fd64800b
[DataFrame] Implemented __setitem__, select_dtypes, and astype ( #1941 )
...
* added reindex, __setitem__, select_dtypes, and astype functionality
* readded tests for astype and select_dtypes
* fixed index issue with reindex
* lint spacing
* removing current reindex implementation for future pr
* wrong testing func
* errors now raised in the workers, but suppressing them can be an issue
* updated code for select_dtypes
* Update test_dataframe.py
2018-05-05 20:27:29 -07:00
8509a51291
[DataFrame] Implement diff ( #1996 )
...
* added diff method
* sanity checks
* flake8
* updated sanity checks'
* rebase and style updates
* updated diff tests and cleaned up code
* updated tests for periods
* flake8
2018-05-05 00:44:52 -07:00
a58629f53f
[DataFrame] Implemented nunique, skew ( #1995 )
...
* implemented prod/product, modified declaration for sum, added pandas test suite
* fixed tests
* removed test_analytics file
* implemented nunique, skew
* fixed changes for nunique, skew
* fixed nunique test
* added axis=1 test to skew
* flake8 issues
* more flake8 issues
* resolve some flake issues
2018-05-04 12:22:10 -07:00
4030356b51
[DataFrame] Implements filter and dropna ( #1959 )
...
* implement filter
* begin implementation of dropna
* implement dropna
* docs and tests
* resolving comments
* resolving merge
* add error checking to dropna
* fix update inplace call
* Implement multiple axis for dropna (#13 )
* Implement multiple axis for dropna
* Add multiple axis dropna test
* Fix using dummy_frame in dropna
* Clean up dropna multiple axis tests
* remove unnecessary axis modification
* Clean up dropna tests
* resolve comments
* fix lint
2018-05-04 12:21:16 -07:00
22d4950fae
[DataFrame] Implements df.pipe ( #1999 )
...
* Add empty df test
* Fix flake8 issues
* rebase with master
* reset master tests
* Implement df.pipe
* fix tests
* Use test_pipe as a pytest.fixture
* Add newline at EOF
2018-05-04 10:16:05 -07:00
a1d7bb31a4
[DataFrame] Apply() for Lists and Dicts ( #1973 )
...
* working for non-string functions and not lists of functions
* works with functions as strings now as well
* fixed linting errors
* throwing a warning if the input is a dictionary
* added dict of lists functionality
* fix minor indexing errors and lint
* removed some commented out code
* some comments and thoughts for apply
* cleaned up code a little bit and added todos
* improved performance
* error checking and code cleanup and comments
* small change
* improved list performance a lot
* agg calls apply for lists
* addressing comments on the PR
* col_metadata change
* updated tests to expect TypeError where appropriate
2018-05-04 10:05:00 -07:00
cdf94c18a4
Clean up syntax for supported Python versions. ( #1963 )
...
* Use set/dict literal syntax
Ran code through [pyupgrade](https://github.com/asottile/pyupgrade ). This is
supported in every Python version 2.7+.
* Drop unnecessary string format specification
No need to specify 0,1.. if paramters are passed in order.
* Revert "Drop unnecessary string format specification"
This reverts commit efa5ec85d30ff69f34e5ed93e31343fea7647bcb.
* Undo changes to cloudpickle
Drop use of set literal until cloudpickle uses it.
* Reformat code with YAPF
We need to set up a git pre-push hook to automatically run this stuff.
2018-05-03 07:45:11 -07:00
d85ee0bc04
[DataFrame] Implements mode, to_datetime, and get_dummies ( #1956 )
...
* implement mode and fix getitem
* mode broken on misaligned partitions
* fully implement mode
* implement to_datetime
* implement get_dummies
* implement tests
* fix __getitem__
* fix python2 compatibility
* fix getitem bug
* resolving comments
* Adding documentation
* resolving comment
* resolve name change
* speeding up getitem
* complete rebase
2018-05-02 23:21:00 -07:00
d67b786291
[DataFrame] Fix dtypes ( #1930 )
...
* Add map, reduce, merge_dtypes
bug fixes
Unify dtypes on DataFrame creation
Formatting and comments
Cache dtypes
Fix bug in _merge_dtypes
Fix bug
Changed caching logic
Fix dtypes issue in read_csv
Invalidate dtypes cache when inserting column
Simplify unifying dtypes and improve caching
Fix typo
Better caching of dtypes
Fix merge conflicts
* Correct dtypes on initialization
2018-05-02 23:04:19 -07:00
4badc04bb2
[DataFrame] Add layer of abstraction to allow OID instantiation ( #1984 )
2018-05-02 22:29:52 -04:00
5589426484
[DataFrame] Fix blocking issue on _IndexMetadata passing ( #1965 )
...
* metadata passing fixes
* fix flake8
* fix test failures
* overhaul indexmetadata
* variable name change
* optimization for building coord df
* addressing comments
* subtle bug fixes
2018-05-01 23:27:49 -07:00
7c1d569a49
[DataFrame] Implement df.merge ( #1964 )
...
* Begin merge implementation
* Some cleanup
* Continue cleanup
* Allowing merge on index
* Copy dataframes to clear plasma read-only error
* Make some notes, WIP
* Cleaned up code a bit, still need more error checking
* Adding error checking and addressing comments
* Addressing comment
* Adding test
* Addressing rebase artifact
* Fixing indexing bug
* Some minor cleanup
2018-05-01 21:40:53 -04:00
1231aa0582
[DataFrame] Sample implement ( #1954 )
...
* implemented sample - need to test
* sample fully working
* added sanity check tests
* added some comments to clarify the _deploy_func call
* some more clarifying comments
* added explanatory comments
* minor change in weights_sum for sample
2018-04-30 10:42:28 -07:00
0c477fbbca
[DataFrame] Implement Inter-DataFrame operations ( #1937 )
2018-04-30 06:42:07 -07:00
1d1df7bbec
[DataFrame] Fully implement append, concat and join ( #1932 )
2018-04-23 17:09:57 -07:00
Devin Petersohn